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A system  of  n components  under  the  care  of  one  repairman 
Is  modeled.  The  components  are  subject  to  failure,  where- 
upon they  may  be  repaired  one  at  a time.  It  is  desired  to 
repair  failed  components  in  such  a manner  that  the  ergodic 
probability  that  the  system  works  is  maximized. 

It  is  assumed  that  each  component  and  the  system  as  a whole 
can  be  either  working  or  failed,  with  the  relationship 
between  the  working  of  the  system  and  the  working  of  the 
components  given  by  a coherent  structure  function.  The  time 
a component  works,  or  the  time  to  repair  a component  is  an 
exponential  random  variable  of  known  rate.  All  components 
are  independent,  and  at  most  one  component  may  be  under 
repair  at  a given  time. 

Although  the  general  problem  is  in  principle  soluble  by 
known  methods,  computational  difficulties  are  enormous  for 
moderate  sized  systems.  In  addition,  such  methods  give  no 
general  Insight  into  the  structure  of  the  optimal  policy. 
Therefore,  bounds  and  approximations  for  general  systems  are 
highly  useful. 

One  bound  for  the  optimal  ergodic  probability  that  the 
system  works  is  given  by  the  ergodic  probability  that  the 
system  works  under  a particular  policy.  The  time  reversible 
policy  given  yields  easily  obtainable  ergodic  probabilities 
for  all  states,  and  is  useful  for  bounding  purposes. 


Most  real  systems  are  highly  reliable  in  nature.  Parametri- 
zation  of  the  rates  of  the  exponential  random  variables 
given  earlier  allows  investigation  of  asymptotic  system 
properties  as  the  system  becomes  very  reliable.  Specifi- 
cally, for  a given  policy,  the  asymptotic  ergodic  probability 
of  all  states  and  the  asymptotic  passage  times  between  states 
may  be  computed.  These  results  allow  one  to  obtain  the 
asymptotic  optimal  unreliability  of  an  arbitrary  system,  and 
to  obtain  the  asymptotically  optimal  policy  for  assignment  of 
the  repairman  in  many  cases.  Intuitively,  the  asymptotically 
optimal  policy  is  close  to  optimal  for  highly  reliable 
systems. 

Although  highly  unreliable  systems  occur  less  frequently, 
such  systems  may  be  treated  in  a similar  manner  with  similar 
results. 

Two  specific  examples  of  systems  are  treated  in  the  paper: 
the  series  system  and  an  arbitrary  system  of  stochastically 
identical  components.  The  series  system  occurs  often  in 
practice,  since  many  simple  systems  cannot  tolerate  failure 
of  any  components. 

— C— 
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These  two  examples  lead  to  relatively  simple  solutions. 
Arbitrary  systems  generally  do  not.  The  two  component 


parallel  system  yields  a fairly  complicated  criterion  for 
choosing  between  the  two  potentially  optimal  policies. 

When  the  series  system  is  composed  of  components  whose 
failure  rates  are  identical,  the  ergodic  probability  that 
the  system  works  is  Independent  of  policy. 

For  the  two  component  series  system,  it  is  optimal  to  repair 
the  longer  expected  lifetime  component  first,  and  this  is 
true  even  if  rhe  repairman  is  subjected  to  random  intervals 
during  which  t >■.  is  not  allowed  to  work. 
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For  an  n component  series  system,  the  optimal  policy  seems 
to  be  to  repair  the  components  in  order  of  increasing 
expected  lifetimes.  This  result  can  be  proven  if  the 
optimal  policy  can  be  written  as  a list,  but  a more  general 
proof  seems  to  be  elusive. 


When  a system  is  composed  of  stochastically  identical 
components,  it  is  often  possible  to  eliminate  most  policies 
from  consideration.  Two  examples  of  this  technique  are 
given,  including  one  in  which  the  optimal  policy  is 
explicitly  obtained. 
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CHAPTER  1 
Prel iminaries 
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1.1  Introduction 


The  field  of  reliability  theory  has  seen  great 
activity  recently.  A central  assumption  to  many  models  is 
the  binary  state  assumption;  that  is,  that  each  component 
and  the  system  as  a whole  can  assume  two  states:  working 

and  failed.  Analysis  of  the  relationship  between  the  sys- 
tem's state  and  the  state  of  the  components  under  reasonable 
assumptions  is  the  subject  cf  coherent  structure  theory.^ 
Given  coherent  structure  theory,  it  is  easy  to 
obtain  deterministic  and  probabilistic  models  for  the  wear 

out  of  a system  when  the  wear  out  characteristics  of  each 

7 

component  are  known. 

An  extension  of  the  probabilistic  model  of  the 

last,  paragraph  treats  a system  whose  components  fail  and 

39 

are  then  repaired.  Ross  treated  a maintained  system  in 
which  each  component  fails  and  is  repaired  again  in  accord- 
ance with  an  alternating  renewal  process.  Keilson  and 

others1  ' ' ' deal  with  the  same  model  under  the  more 

th 

specific  assumption  that  the  time  that  the  i—  component 
works  and  the  time  required  to  repair  the  i—  component  are 
exponential  random  variables.  Barlow"’  treats  a system  where 
other  components  are  in  a state  of  "suspended  animation" 
during  repair  of  a component.  All  of  these  maintained  sys- 
tem models  approximate  a system  which  has  separate  repair 
resources  dedicated  to  each  component.  A more  realistic 
assumption  for  some  systems  is  that  the  system  possesses 

— — - .7  i . 

... . - 


limited  repair  resources  which  may  be  allocated  to  failed 
components  as  needed.  We  attempt  to  model  such  a system. 
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In  the  simplest  case,  the  system  possesses  linear- 
ity of  repair  resources  and  no  comparative  advantage  in  the 
repair  of  different  components  by  different  resources.  As 
will  be  shown  in  section  2.1,  all  repair  resources  may  be 
optimally  concentrated  on  a single  component  at  any  time. 
Thus,  in  this  case,  the  system  may  be  thought  of  as  having 
a single  repairman  who  can  repair  at  most  one  component  at 
a given  time.  This  model  will  be  the  principal  object  of 
investigation  in  this  paper. 

Classical  repairman  models2 '3'4'8'10'20f22»31»33'41 
deal  with  a system  of  components  and  spares.  Upon  failure, 
a component  is  replaced  with  a working  spare,  if  available, 
and  then  sent  into  queue  at  the  repair  facility  to  be  re- 
paired. Usually,  only  one  working  component  is  treated  in 
such  a model. 

f..  ...  . 

_ V . » 

For  our  model,  if  general  failure  and  repair  dis- 

39 

tnbutions  are  allowed,  as  m Ross,  things  become  very 

complicated.  Such  a model  needs  Markovian  decision  theory 

on  uncountable  state  spaces,  with  all  the  attendant  diffi- 

24 

culties.  Therefore,  as  in  Keilson,  we  assume  that  all 
failure  and  repair  distributions  are  exponential.  The 
memoryless  property  of  the  exponential  distribution  allows 
us  to  specify  the  system  state  solely  in  terms  of  the  binary 
states  of  all  components. 

In  section  1.2  we  describe  the  assumptions  and 


| 
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results  of  coherent  structure  theory  appropriate  to  our 
model,  which  is  described  in  section  1.3.  Our  objective 
will  normally  be  to  maximize  the  ergodic  probability  that 
the  system  works.  Section  1.4  proves  the  intuitive  and  im- 
portant result  that  it  is  not  optimal  to  leave  the  repair- 
man idle. 

General  results  for  the  model  are  often  difficult 
to  obtain.  Division  II  describes  three  methods  for  approx- 
imating and  boundary  optimal  system  characteristics. 

In  Chapter  2 we  find  a particular  policy  for  which 
the  ergodic  probabilities  are  easily  obtainable.  The  opti- 
mal performance  is  bounded  by  the  performance  of  any  given 
policy,  and  in  particular,  is  bounded  by  the  performance  of 
the  policy  given  in  Chapter  2.  In  addition,  the  coordinates 
appear  to  be  associated  under  the  above  given  ergodic 
probabilities,  allowing  a further  simplification  in  the 
computation  of  a system  bound. 

Chapters  3 and  4 deal  with  asymptotic  results  for 
highly  reliable  or  unreliable  systems  respectively.  These 
results  are  obtained  by  assuming  that  the  failure  rates  or 
repair  rates  of  all  components  are  multiplied  by  a common 
factor  k,  and  looking  at  small  k.  The  ergodic  probabilities 
of  states  under  arbitrary  policies  are  asymptotically  pro- 
portional to  integral  powers  of  k,  allowing  computation  of 
the  asymptotic  optimal  unreliability  or  reliability  of  the 
system  in  terms  of  specific  constants  multiplied  by  powers 
of  k.  An  alternating  renewal  theory  approach  then  allows 
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one  to  find  an  easy  to  use  optimization  procedure  which 
often  determines  the  action  which  is  asymptotically  optimal 
in  a given  state. 

The  third  division  of  the  paper  deals  with  various 
specific  systems.  The  general  complicated  nature  of  the 
problem  of  deciding  the  optimal  policy  is  illustrated  with 
the  solution  of  a two  component  parallel  system. 

We  then  treat  a series  system  in  which  the  compon- 
ents have  identical  failure  rates.  The  somewhat  surprising 
conclusion  is  that  the  policy  does  not  matter.  The  ergodic 
probability  that  the  system  works,  the  Laplace  transform  of 
the  time  to  fix  the  system  from  any  state  and  the  expected 
integral  of  the  discounted  time  the  system  works  are  all 
obtained. 

For  the  series  system  with  two  components,  the 
optimal  policy  is  to  repair  the  component  with  smaller  fail- 
ure rate  first.  This  is  true  even  if  the  repairman  is  sub- 
ject to  interruptions;  intervals  of  time  in  which  he  is  not 
allowed  to  work.  This  result  shows  that  if  the  optimal 
policy  of  the  series  system  is  in  the  form  of  a list,  the 
components  must  be  repaired  in  order  of  increasing  failure 
rate.  A policy  of  such  a form  is  intuitive  for  the  series 
system,  but  no  proof  is  yet  available. 

We  next  treat  a system  made  of  stochastically 
identical  components.  Often,  symmetries  in  the  structure 
function  show  that  certain  actions  in  certain  states  are 
non-ogtimal.  Sometimes  enough  actions  can  be  ruled  out  to 
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actually  give  the  optimal  policy.  Several  examples  of  the 
use  of  this  technique  are  given. 
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1.2  Coherent  Structure  Theory 
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One  of  the  common  assumptions  for  reliability 

7 

models  is  the  existence  of  a coherent  structure.  We  in- 
clude these  assumptions  for  our  model. 

Specifically,  we  assume  that  the  system  is  made 
up  of  n components,  and  that  the  system  itself  and  each  com- 
ponent can  be  in  one  of  two  states,  functioning  or  failed. 
Thus  the  state  of  the  n components  can  be  summarized  by  a 
binary  n-vector  X with  the  understanding  that  » 1 iff  the 
i—  component  is  functioning,  and  that  Xi  * 0 iff  the  i— 
component  is  failed.  This  vector  X will  be  called  the  state 
of  the  system. 

We  further  assume  that  whether  the  system  is  func- 
tioning or  failed  is  a function  only  of  the  states  of  the  n 
components,  or  equivalently,  if  x represents  the  set  of  all 
binary  n-vectors. 


3 X 

♦ (X)  - 1 


♦ (X) 


-*•  {0,1}  , s.t.  , 

iff  the  system  is  functioning  when 

the  system's  state  is  X, 

iff  the  system  is  failed  when  the 

the  system' 8 state  is  X. 


The  function  <p  is  called  the  structure  function. 

It  is  reasonable  to  assume  that  4>  is  monotone  non* 
decreasing,  corresponding  to  the  intuitive  notion  that 
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repair  of  a component  cannot  cause  system  failure. 

A component,  i,  is  relevant  if  0(1^,X)  ¥ 0(O^,X) 
for  some  X.  Here  the  notation 

1^/X  = (X^ , • • • , X^_^ , 1 ,X^ ^ , • • • , X^ ) , 

and  0 ^ , X = (X^m*«»X^_^»0,X^^,.»«i  X^ ) • 

Without  loss  of  generality  we  need  only  consider  relevant 
components . 

A structure  function  that  is  non-decreasing  and 
for  which  all  components  are  relevant  will  be  called  coher- 
ent. 

Given  a coherent  structure  function  <p , X is  a path 
vector  if  (X)  = 1.  The  corresponding  path  set  is  (X) 

{ i | Xi=l } . A path  set  is  minimal  if  no  proper  subset  of  the 
path  set  is  a path  set.  It  is  easily  seen  that  0 (X)  =1  if 
and  only  if  some  minimal  path  set  is  a subset  of  C^(X). 

Every  component  must  be  in  at  least  one  minimal  path  set. 

Similarly  a vector  X is  a cut  vector  if  0 (X)  * 0. 
The  corresponding  cut  set  is  Cg (X)  {i|X^=0}.  A cut  set 

is  minimal  if  no  proper  subset  of  the  cut  set  is  a cut  set. 
It  is  easily  seen  that  0 (X)  =0  if  and  only  if  some  minimal 
cut  set  is  a subset  of  Cg(X).  Every  component  must  be  in 
at  least  one  minimal  cut  set. 
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1 . 3 The  Model 

We  deal  with  an  n component  system  with  a coherent 
structure  function  $.  We  assume  that  the  evolution  of  the 
state  vector;  X,  is  governed  by  the  following  rules: 

1.  The  i~  component  functions  for  a random  per- 
iod of  time  exponentially  distributed  with  rate  > 0. 

After  such  a period  of  time,  it  fails. 

2.  The  i—  component,  when  failed  and  under  repair, 
is  repaired  (goes  from  failed  to  functioning  state)  in  a 
random  period  of  time,  exponentially  distributed  with  rate 

Ai  > 0.  When  the  i—  component  is  failed  and  not  under  re- 
pair it  does  not  change  state. 

3.  The  component  evolutions  (given  the  component 
under  repair)  are  independent  of  each  other  and  the  previous 
history  of  the  system. 

We  also  assume  the  following  two  rules  for  assign- 
ment of  the  repairman. 

4.  The  component  under  repair  may  be  changed 
arbitrarily  quickly. 


5.  At  most,  one  component  may  be  under  repair  at 
a given  time.  Note  that  we  have  the  option  of  leaving  the 
repairman  idle. 

A policy  will  be  defined  to  be  any  rule  for  assign- 
ment of  the  repairman.  A policy  is  a stationary  pure  policy 
if  the  repairman  is  assigned  by  a deterministic  function  of 
state  only.  For  a stationary  pure  policy  tt,it(X)  will  denote 
the  component  under  repair  in  state  X. 
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Define  F to  be  the  expected  integral  of  the  dis- 
counted time  the  system  is  working.  Thus  for  S>0,  f an 
arbitrary  policy,  and  X(t)  the  system  state  X at  time  t. 


F(n,S,  e ^ (X(t)  )dt|X(0)=X0 


(1.3.1) 


Standard  results'3  tell  us  that  there  exists  a 
stationary  pure  policy  tj*,  such  that 


F (it  * , S , Xq  ) 


sup  F (tt  ,S  , X_) 

TT  “ 0 


(1.3.2) 


The  same  tt*  satisfies  (1.3.2)  regardless  of  X^. 
When  the  state  space  is  finite,  as  in  our  model, 
a stationary  pure  policy  tt*  exists,  such  that  (1.3.2)  is 
true  for  all  X^  and  S<SQ  with  SQ>0.  Furthermore, 


lim  SF(Ti*,S,Xn) 
S-0 


X : <f>  (X)  =1 


(1.3.3) 


where  e^atX)  is  the  ergo  lie  probability  of  the  state  X 
under  policy  it*.  Thus  to  find  a policy  which  maximizes 
F(tt,S,X^)  over  tt  for  small  valves  of  S,  we  need  only  find  a 
stationary  pure  policy  which  maximizes  the  ergodic  probabil- 
ity that  the  system  works.  Our  objective  in  the  following 
paper  will  normally  be  to  maximize  the  ergodic  probability 
that  the  system  works  over  the  stationary  pure  policies 
possible . 
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1.4  Strict  Optimality  of  Assignment  of  Repairman 

In  this  section  we  prove  the  intuitive  notion  that 
the  repairman  should  never  be  idle  unless  all  components  of 
the  system  are  working. 

Theorem  1.4  A.  Let  it'  be  a stationary  pure  policy 
in  which  the  repairman  is  idle  in  at  least  one  state  which 
is  not  1.  Then  at  least  one  state  X*  / 1 with  the  repair- 
man idle  must  be  positive  recurrent  under  the  Markov  chain 
which  describes  the  system  evolution  under  tt 

Proof;  Clearly  0 is  positive  recurrent  since  all  > 0, 
and  since  there  are  a finite  number  of  states. 

If  the  theorem  is  not  true,  then  some  component  j 
is  under  repair  in  the  0 state.  Since  > 0,  we  obtain 
1^,0  is  positive  recurrent.  Repeating  this  argument  n times 
tells  us  that  1 is  positive  recurrent. 

But  again  > 0 implies  all  states  are  positive 
recurrent,  a contradiction.  End  of  proof  of  Theorem  1.4  A. 

Theorem  1.4  B.  A stationary  pure  policy  n * which 
leaves  the  repairman  idle  in  some  state  not  1 does  not 
maximize  F(tt,S,Xq)  for  any  value  of  S or  any  Xq. 

Proof : We  contradict  the  optimality  of  it'  by  comparing  it 

with  a derived  policy  tt".  The  evolution  of  the  state  under 
n',  X*  (uj , t)  • and  under  w",  x"(w,t);  will  be  mapped  into  the 
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same  probability  space  in  such  a way  that  X"(u),t)  >_  X'  (w,t)  , 

V w,t.  Thus,  by  the  monotonicity  of  <p  , 4>  (X"  (w,  t) ) >_ 

<p  (X'  (u>,  t) ) , V oo,  t.  Furthermore  we  show  that  <MX"(w,t))  * 1 
and  <MX'  (w,t))  = 0 on  a set  large  enough  to  contradict  the 
optimality  of  tt  ' . 

Let  (ft,?,P)  support  two  processes  X'  and  Y with 
the  following  joint  distribution:  Let  X'  evolve  according 

to  policy  if  * with  X'  (w,0)  = Xq.  Let  6 be  the  first  time  that 
X'  reaches  the  state  X*  whose  existence  is  guaranteed  by 
Theorem  1.4  A,  let  t be  time  spent  in  X*  before  the  first 
failure . Conditional  on  X'  , let  Y describe  the  system  evolu- 
tion with  Y (w , 0 ) = 0 under  the  following  policy:  before  time 

x repair  the  smallest  numbered  component  in  Cq(X*)  which  is 
failed  in  Y (if  none  are  failed,  leave  the  repairman  idle) 
and  after  time  t let  the  repairman  be  idle. 

Now  define  X"  on  (ft,?, P)  as  follows: 


X"  (t)  = X'  (t)  , 0 _<  t < i , 

X"  (t)  = X'  (t)  V £<t-6)  Y(t-6)  , 6 <t<“>, 


(1.4.1) 


where  (X  V Y)  i = max(Xi,Yi)  , and  where  A(t-5)  is  a diagonal 
matrix  with  Aii(tQ-6)  = 0 if  X|(t)  = 1 for  some  6 < t < tQ  % j 
1 otherwise.  My  thanks  to  Jamos  Pitman  who  considerably 
shortened  the  original  argument  to  this  point. 

It  is  clear  that  X"  is  the  proper  system  evolution 
under  a policy  which  uses  policy  tt  * until  X*  is  reached,  then 
repairs  the  lowest  numbered  failed  component  until  there  is 


f 
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a failure  in  C^(X*),  and  then  repairs  the  component  tt  • (x* ) 
(unless  this  component  is  working,  in  which  case  the  repair- 
man is  idle) . This  is  some  non-stationary  probabilistic 
policy. 

From  (1.4.1)  X"  (t)  >_  X'  (t)  . We  now  show  strict 
domination  for  each  of  two  cases. 

Case  1.  $(X*)  "0.  We  note  that  there  is  non-zero 

probability  that  under  nM,  all  components  in  CQ(X*)  are  re- 
paired in  the  interval  (6,6+r).  This  shows  strict  domination 
since  $(1)  >1  for  a coherent  structure. 

Case  2.  $(X*)  * 1.  Pick  a component  jeCQ(X*)  and 

let  a minimal  cut  set  containing  j be  K.  Such  a minimal  cut 
set  can  always  be  found  for  any  component  of  a coherent 
structure.  Let  K*  = K fi  C^X*). 

We  know  that  there  is  non-zero  probability  that 
only  the  components  in  K*  fail  without  repairs  occuring 
for  X'  in  an  interval  beginning  at  6,  while,  during  the  same 
interval,  all  components  in  CQ  (X*)  are  repaired  for  X"  with- 
out failures  occuring  for  these  components. 

In  this  case  <MX')  = 0 since  K is  a subset  of 
the  failed  components  of  <P  (X * ) , and  $(XM)  * 1 since  the 
failed  components  of  X"  are  a proper  subset  of  a minimal  cut 
set  K.  End  of  proof  of  Theorem  1.4  B. 

Corollary : A stationary  pure  policy  which  does  not  assign 

the  repairman  in  a state  not  1^  cannot  achieve  the  maximum 


Thus,  in  optimizing  the  ergodic  probability  that 
the  system  works,  we  need  only  consider  candidate  policies. 

Arguments  similar  to  those  used  in  the  proof  of 
Theorem  1.4  A tell  us  that  the  system  evolution  under  a 
candidate  policy  is  governed  by  a finite,  irreducible  con- 
tinuous time  Markov  chain. 
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ergodic  probability  that  the  system  works. 


Definition : A stationary  pure  policy  which 

assigns  the  repairman  in  all  states  ^ 1 will  (1.4.2) 

be  called  a candidate  policy. 


DIVISION  II 
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Approximations  and  Bounds 
CHAPTER  2 

The  Time  Reversible  Policy 
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2 . 1 Simultaneous  Effort  and  Mixed  Policies 

Suppose  that  we  allow  the  repairman  to  simultan- 
eously expend  a portion  of  his  effort  on  several  failed 
components.  If,  while  in  state  X,  he  expends  a proportion 
of  his  effort  p^(X)  on  the  i—  failed  component,  it  is 
reasonable  to  assume  that  the  i—  component's  repair  rate 
is  then  P^(X)A^,  with  failure  rates  remaining  unchanged. 

By  argumer.ts  similar  to  those  advanced  in  section  1.4,  if  a 
policy  is  such  that  the  repairman  does  not  expend  all  his 
effort  in  a given  state,  the  system  evolution  under  that 
policy  is  dominated  by  the  system  evolution  under  another 
policy  which  expends  all  the  repairman's  effort  in  the  same 
state.  Also,  standard  results  tell  us  that  an  optimal  policy 
is  a stationary  policy. 

Thus,  in  finding  optimal  policies  when  allowing 
the  repairman  the  flexibility  of  simultaneous  effort  de- 
scribed in  the  previous  paragraph,  we  need  only  consider 
candidate  simultaneous  effort  policies  defined  below. 

(2.1.1)  Definition:  A candidate  simultaneous 

effort  policy  is  described  by  a set  of  probability  n-vectors 
p (X) , X ^ 1,  with  the  properties: 

a)  p^X)  = 0,  ieC^X)  , 

b)  pi(X)  >0, 
n 

c)  l p.  (X)  = 1 , 
i=l  i 

and  with  the  interpretation  that  the  proportion  of  the 
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repairman' e effort  spent  on  component  i in  state  X is  p^(X). 
Thus,  the  repair  rate  of  component  i in  state  X,  ieC^  (X) 
is  p^(X)X^.  The  failure  rate  of  component  i is  unchanged 
and  is 

Note  that  a candidate  policy  is  a special  case  of 

a candidate  simultaneous  effort  policy. 

Another  model  which  gives  rise  to  restricted 

simultaneous  effort  policies  is  the  following:  the  system 

possesses  m repair  resources,  or  "men."  If  the  i— man  is 

th 

allocated  to  the  j — component,  the  -repair  rate  for  the  j — 
component  is  X^.  We  assume  there  is  no  comparative  advan- 
tage of  men  in  repairing  components,  or  more  precisely, 

X.  ./X.  . is  independent  of  j.  Furthermore,  we  assume  that 

xl3  123 

the  application  of  repair  resources  to  the  system  is  linear, 
or  more  precisely,  if  the  men  in  a set  Ij  are  assigned  to 
fix  component  j , then  the  repair  rate  of  component  j is 


iela 


Thr  fact  that  there  is  no  comparative  advantage 
of  men  in  fixing  components  requires  that 


X . . = r . X . 

ij  i D 


(2.1.2) 


m 

where  r.  can  be  normalized  so  that  E r.  = 1. 

1 i=l  1 

Thus,  in  this  model,  if  we  employ  a stationary 

policy  wherein  the  men  in  the  set  I . (X)  are  assigned  to  re- 

3 ” n 

pair  failed  component  j in  state  X,  with  U la (X)  = 

j = l 3 ~ 

{l,2,...,m}  , X ^ 1,  then  the  system  evolution  is  equivalent 


I 


U'JRP.i 


to  the  system  evolution  under  a candidate  simultaneous 


effort  policy  of  definition  (2.1.1),  with 


p.(X)  = 


(2.1.3) 


iel j (X) 


and  Xj  and  r^  given  by  (2.1.2) 


(2.1.4)  Definition.  A policy,  it,  is  called  a 


stationary  probabilistic  or  mixed  policy,  if  upon  change 


of  the  system  state  to  X,  the  repairman  is  assigned  to  repair 
component  i with  probability  q^(X),  and  is  not  reassigned 


until  the  system  changes  state  again. 


The  following  theorem  provides  a way  to  compare 


ergodic  probabilities  under  simultaneous  effort  policies 


with  ergodic  probabilities  under  mixed  policies, 


Theorem  2.1  A.  Let  e(X)  be  the  ergodic  probability 


of  state  X under  the  Markov  chain  which  describes  the  system 


evolution  under  a policy  given  in  definition  (2.1.1),  and 


let  e' (X,i)  be  the  ergodic  probability  of  state  X with  the 


i—  component  under  repair  in  the  Markov  chain  which  describes 


the  system  evolution  under  definition  (2.1.4).  Furthermore 


let: 


q4  (X)  - P4  (X)  (A.+uMX))/  £ P,  (X)  (A.+UMX))  , ieCn  (X) 

1 ” 1-1  - / jt.c0(x)  3 - 3 u 


otherwise , 


- - — mmn 


19 


where  m*(X)  ■ Z , and  3^  and  £ are  the  3 and  £ 

ieC1(X) 

from  the  appropriate  definitions  qiven  above. 

Then 


e (X)  Z e' (X,i) 

ieCQ (X) 


(2.1.6) 


Proof : All  states  of  the  second  chain  which  are  possible 

to  enter  are  easily  seen  to  be  positive  recurrent. 

Since  the  first  chain  is  ergodic,  it  satisfies 
the  balance  equations. 


[U*(X)+  Z P,  (X)  A.Je(X)-  Z A.p.  (0,  ,X)e(0.  ,X)  + 

ieC0(X)  “ ieC^X)  1 1 1 “ 1 _ 

(2.1.7) 

Z e (1.  , X) 

iec0  (X) 


We  will  show  that 


e'(X,i)  * e(x)pi(X) 


(2.1.8) 


which  proves  the  desired  result.  This  is  done  by  showing 
that  (2.1.8)  satisfies  the  balance  equations  for  the  second 
chain,  which  are: 


(M*(X)+A.)e' (X,i)  - q.  (X)  ( Z A.e'(0.,X,j)  + 

“ jeCi(X)  3 3 


(2.1.9) 


Z Z vi.e'  (1 . ,X,k)  ] 
k jeC0 (X)  3 3 
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Substituting  (2.1.8)  into  (2.1.9),  we  obtain 


p.  (X)  (uMX)+A.)e(X)  = q4  (X)  [ L X,p  . (0  . ,X)  e (0  . ,X)  + 

1 - - l - i - jECl(x)  J 3 J J 


t U^e (1 • ,X) ) . 

jeC0(X)  3 J 


Substituting  (2.1.7)  into  the  square  brackets  gives: 


Pi(X)  (u*(X)+Xi)e(X)  = 


q j ( X ) [ ( u * ( X ) + l p,  (X)X.)e(X)] 

j£C0(X)  J J 


and  equality  in  (2.1.11)  is  guaranteed  by  (2.1.5). 

We  note  that  the  transformation  between  £ and  c[ 
of  (2.1.5)  is  a one  to  one  and  onto  transformation  between 
probability  vectors  defined  to  be  non-zero  on  Cg (X) . End 
of  proof  of  Theorem  2.1  A. 


The  inverse  transformation  of  (2.1.5)  is 


qA(x) 


Pi (x)  = 


jeCo (X) 


qj(X) 


Vi 


We  have  the  following  coilary  to  Theorem  2.1  A: 


pwp"  ' ■ II'  " 1 1 ■■■  | 1 p 
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If  the  repairman  is  allowed  to  expend  simultaneous 
effort  as  in  definition  (2.1.1) , the  ergodic  probability 
that  the  system  works  may  be  maximized  by  maximizing  the 
ergodic  probability  that  the  system  works  over  candidate 
policies.  This  follows  from  Theorem  2.1  A and  the  standard 
result  that  mixed  policies  need  not  be  considered  for  optim- 
ization purposes. 

Thus , for  the  model  of  linear  repair  resources 
without  comparative  advantage  presented  earlier,  all  re- 
sources may  optimally  be  allocated  at  any  time  to  one  failed 
component. 


JL_ 


- 


- ■ 


Bwaan 
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2 .2  The  Time  Reversible  Chain 
Simultaneous  Effort  Polici 


A continuous  time  Markov  chain  is  said  to  be  time 
24 

reversible  if  there  is  equality  of  ergodic  flows  between 
any  two  states.  Thus  we  require  A^e^  = A^e^  f°r  anY  two 
states  i and  j,  where  A^  is  the  rate  of  transitions  from  i 
to  j,  and  is  the  ergodic  probability  of  state  i. 


Theorem  2.2  A.  There  is  a unique  set  of  probabil- 
ity vectors  £(x) , X ^ 1 such  that  p^(X)  = 0,  ieC^(X),  for 
which  the  system  evolution  under  the  policy  described  in 
definition  (2.1.1)  is  governed  by  a time  reversible  Markov 
chain.  This  policy  will  be  called  the  time  reversible 


policy . 


X / 1 by 


Furthermore,  the  unique  £(X)  are  determined  for 


PlU)  |C0(X) 


ieCQ (X) 


otherwise  , 


(2.2.1) 


and  the  ergodic  potentials,  it  (X)  , are 


(X)  = I CQ (X)  | ! 


ieC0  (X)  i 


(2.2.2) 


Here  |Cq(X)|  denotes  the  cardinality  of  Cg(X). 


Proof : Assume  the  existence  of  the  proper  (X)  . Arbitrar- 

ily define  the  potential  of  1 to  be  1, 


"(I) 


(2.2.3) 


We  now  prove  the  theorem  by  induction  on  |CQ(X)|. 
When  | CQ (X) | - 1,  £(X)  must  be  given  by  (2.2.1). 
Also  by  balance  of  ergodic  flows  between  0^,1  and  1^ 


V (<^,1) 


^i71  (i.)  * 


(2.2.4) 


* (0  • # 1)  ■ r — 

1 “ Ai 


which  satisfies  the  theorem. 


(2.2.5) 


Now  assume  that  the  theorem  is  true  for  |CQ(X)|  *= 


m-1.  By  equating  ergodic  flows,  we  obtain: 


L.  P,  (0.  ,...0.  ,1)tt(0,  , . . . ,0.  ,1)  = 

xj  1l  xm  xl  xm  ” 

M . ^ (0  ■ , . . • , 0 • ,0.  ,.«.,()•  ,1)  (2.2.6) 

xj  X1  xj-l  ij+l  xm  ” 


for  j = 1, . . . ,m 


Now,  using  the  induction  hypothesis  that  (2.2.2) 
is  true  for  the  right  hand  side  of  (2.2.6),  we  obtain 


Pj  ( 0 t ,l)n(0.  , • • • , 0 • ,1^) 

xj  X1  xm  ” xl  1m 


(2.2.7) 


m y^ 


(m-1)  1 n (^ 
j=l  ij 
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or  that  all  p.  are  equal,  i.eCn(X) 
1 j 3 u - 

Thus , 


p.  (0.  ,...,0.  ,1) 

j "l 


m 


m 


(2.2.8) 


and 


TT  (0.  ,...,0.  ,1) 

X1  m 


m 1 


m 

II 

j=l 


(2.2.9) 


The  theorem  is  therefore  true  for  |Cq(X)|  = m,  and 
the  induction  is  complete. 

We  have  actually  constructed  a time  reversible 
chain,  and  assuming  its  existence,  shown  its  uniqueness. 

End  of  proof  of  Theorem  2.2  A. 

The  ergodic  probabilities  are  just  the  normalized 
potentials.  Therefore  the  ergodic  probability  that  the 
system  works,  W,  under  the  time  reversible  policy  is 

l TT  (X) 

X : 4>  (X)  =1 

W « (2.2.10) 

L TT  ( X) 

X 

where  tt  (x)  is  given  by  (2.2.2). 

W is  a lower  bound  to  the  optimal  ergodic  reliabil- 
ity of  the  system. 

Theorem  2.1  A tells  us  that  W can  be  achieved  by 
the  mixed  policy  given  in  definition  (2.1.4)  i 

! 

j# 

L 


1 '."in  ipi  "• 


A.+U* (X) 


qi(X) 
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(2.2.11) 


E (X.+M*(X)) 
jeCQ (X)  J 


for  ieCQ  (X)  . 


Of  course  the  system  evolution  in  this  case  is  not 
given  by  a time  reversible  chain. 


* 


2.3  Association  of  the  Components 


Under  the  Ergodic  Probabilities  of 


the  Time  Reversible  Policy 


The  bound,  W,  of  the  previous  section  is  difficult 
to  compute  if  the  number  of  components  in  the  system  is  sub- 
stantial, since  the  number  of  states  is  equal  to  2n,  where 
n is  the  number  of  components. 

The  computation  of  a bound  for  the  ergodic  prob- 
ability that  the  system  works  could  be  simplified  a great 
deal  if  the  Esary-Proschan-Walkup  cut  set  bound^  is  appro- 
priate. This  requires  association  of  the  components.  It  is 
conjectured  that  when  the  state  probabilities  are  given  by 
the  normalized  potentials  of  (2.2.2),  the  components  are 
associated.  However,  only  the  association  of  any  two  com- 
ponents is  proven  in  this  section. 

Definition: ^ Random  variables  Ti'***»Tn  are 
associated  if  cov  [T  (T) , A (T)  ] >_  0 for  all  pairs  of  increasing 
binary  functions  T,  A. 

Therefore,  the  components  are  associated  if: 


E[T  (X)  A (X)  ] > E[T(X)  ]E[A(X)  ] 


(2.3.1) 


for  all  increasing  binary  functions  r,A.  (2.3.1)  is  equi- 
valent to 


P [ r (x)  =i, a (x)  =U  i p[r(x)=i]p[A(x)=i] 


(2.3.2) 
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Therefore/  using  (2.2.2)/  we  have  that  the  coor- 
dinates are  associated  under  the  ergodic  probabilities  of 
the  time  reversible  policy  of: 


E TT  (X) 

> T 

E TT  (X) 

X 

X:  T(X) =1 

, . 

A { X)  *1 

(2.3.3) 

E TT  (X) 

X: A (X) =1  ' 


E TT  (X) 
X : r (X) =1 


for.  all  increasing  binary  functions  r and  A.  (2.3.3)  is 
equivalent  to 


* 

E tt(X) 

E TT  (X) 

X:  T (X) =0 

X:  T (X)  =1 

A (X) =0 

A (X) *1 

, t 

E tt(X) 
X: T (X) =1 
A (X) =0 


1 


J 


Z w (X) 
X:  A (X)  =1 

r (x) =o 


(2.3.4) 


Theorem  2.3  A.  When  tt  (X)  is  given  by  (2.2.2), 

(2.3.4)  is  true  for  all  binary  increasing  functions  T and  A 

i 

of  the  coordinates  X.  and  X.. 

| 


Proof:  If  the  right  hand  side  of  (2.3.4)  is  zero,  the 

inequality  is  satisfied,  since  all  terms  on  the  left  hand 
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side  are  positive. 


The  only  binary  increasing  T and  A which  give  a 
non-zero  right  hand  side  are  T = X^,  A = Xj  or  vice  versa. 
Thus  we  need  show  that 


l TT  (X) 


X : X . =0 

X =0 
3 


£ TT  (X) 


X: X. =1 

" Xj=° 


Z 7T(X) 


X: X . =1 
-x‘=l 


Z TT  (X) 


X:  X . =0 

X1  = l 
3 


(2.3.5) 


We  accomplish  this  through  the  inequality 


tt  (0i,0  • ,X)ir(li,lj,Y)  + TT(0i,0j,Y)it(li,ljfX)  > 

(2.3.6) 


n(0i,  ljfX)7T  (li,0j,Y)  + TT(0i,lj,Y)TT(li,0j,X) 


When  (2.3.6)  is  summed  over  all  X and  Y we  obtain 
32  times  (2.3.5)  . 

To  prove  (2.3.6)  let  a = lco^i'^j'— H 

b = I CQ  ( lj_ » 1 j » X)  I 

Then  by  (2.2.2),  (2.3.6)  reduces,  after  division  by  a common 
factor,  to 


a ! (b+2 ) ! + b 1 (a+2 ) ! > 2 ( a+1 ) ! (b+1) ! 


(2.3.7) 


iwnumi  umvtr'fr 
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which  is  equivalent  to 


b+2  a+2 

a+1  b+1 


> 


2 


(2.3.8) 


which  is  readily  verified  to  be  true  for  a,b  >_  0. 
proof  of  Theorem  2.3  A. 


End  of 


4 


Actually,  (2.3.4)  seems  to  be  true  for  T and  A 
arbitrary  binary  increasing  functions  of  all  coordinates, 
but  the  proof  seems  rather  involved. 


1. 


Approximations  of  highly  reliable  systems  are 

.....  th 

treated  in  the  following  chapter.  We  assume  that  the  1 
component's  failure  rate  is  kuj.  and  the  i—  component's 
repair  rate  is  As  k goes  to  zero,  we  wish  to  know  the 

optimal  policy,  and  also  the  approximate  optimal  unreliabil- 
ity of  the  system.  An  example  of  the  application  of  the 
results  obtained  in  this  chapter  to  a highly  reliable  system 


A 


3.1  Limiting  Ergodic  Probability  of  States 


for  Candidate  Policies 


Theorem  3.1  A.  Let  the  failure  rate  of  the  i — 
component  be  ky^  and  the  repair  rate  of  the  i—  component 
be  Let  n be  a candidate  policy  (see  section  1.4),  and 
e^(X,k)  be  the  ergodic  probability  of  state  X under  policy 
ir . Then 


lim  (k1- I C0 (-) I)  e (X,k) 
k-0  " “ 


(3.1.1) 


Proof:  The  result  is  proven  by  induction  on  | CQ (X) | . 

Clearly  (3.1.1)  holds  for  |C0 (X) | - 0,  since 
0 < e (X,k)  < 1.  Now  assume  (3.1.1)  is  true  for  I Cn (X) I = 

- TT  - — I 0 _ . 

0 / • • • f m<  xi  • 

When  we  equate  the  ergodic  rate  of  entering  and 
leaving  the  group  of  states  for  which  |Cq(X)  j £ m,  we  obtain 


X: | C0 (X) | =m+l 


X:  |CQ  (X)  |=i 


e (X,k)  Z ky. 

=m  * ieCx(X) 

(3.1.2) 


Multiply  both  sides  of  equation  (3.1.2)  by  k and 
take  the  limit  as  k goes  to  zero.  By  the  induction  hypothe- 
sis, the  right  hand  side  has  a limit  of  zero  as  k goes  to 


zero. 


Since  the  left  hand  side  is  a sum  of  non-negative 
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terms,  the  limit  of  each  term  as  k 0 must  be  zero.  The 
induction  is  established  since  ^ 0#  V X s.t.  | Cq  (X)  | * 

m + 1 > 0.  End  of  proof  of  Theorem  3.1  A. 


For  the  remainder  of  the  chapter,  we  suppress  the 
dependence  of  e^  on  k. 

Theorem  3.1  B.  Under  the  conditions  of  Theorem 

3.1  A, 


lim  k I c0  I e (X)  = g (X)  , (3.1.3) 

in  TT  — IT  ~ 

k-»0 


with  0 < g^fX)  < °°»  and  9^  (1.)  = 1 • 


= 2 * * * I • 0.1.4) 

77  ieCQ  (X)  71 


Thus,  (X)  can  be  computed  recursively  for  in- 
creasing | C Q (X)  | . 


Proof:  Again  use  induction  on  |Cq(X) |.  We  note  that 

Ee  (X)  = 1.  Theorem  3.1  A tells  us  that  lim  e(X)  = 0, 

X 77  " k-0  17  “ 

X ft  1.  Thus,  lim  e (1)  = 1,  establishing  the  theorem  for 
k-0  77 

|C0(X) | = 0. 

Now  assume  that  the  theorem  is  true  for  all  X 


such  that  |Cq(X)  j = m. 

Equating  the  ergodic  rate  of  leaving  and  entering 
a state  X yields:  * 
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(X  (X)  + k E uJelX) 

” 1 — / w\  A IT  — 


ieCL(X) 


(3.1.5) 


E X,ew(0.,X)+k  E U<e(l,,X)  . 

ieCi  (X)  1 11  1 " ieCn  (X)  1 * 1 “ 

i=ir(Oi,X)  u " 


Assume  I Cq (X) | s m+1.  Multiply  both  sides  of 

(3.1.5)  by  Since  ieC^  (X)  implies  |Cq(0^,X)|  ■ m+2. 

Theorem  3.1  A tells  us  all  terms  in  the  first  summation  on 

the  right  hand  side  have  zero  limit. 

Also  ieCg(X)  implies  that  |Cq(1^,X)|  = m,  so  that 

the  induction  hypothesis  tells  us  the  limit  of  the  second 

sum  on  the  right  hand  side  is  E y . g (X) . 

ieC0 (X) 

Theorem  3.1  A tells  us  the  second  term  on  the  left 
hand  size  has  zero  limit.  We  conclude  that 


lim  ]C"(m+1,i  e (X)  =■  l u.g.Uj.X) 

k-o  *(a>  * ieC0(X)  1 * 1 " 

thus  establishing  (3.1.3)  and  (3.1.4)  for  |CQ(X)|  = m+1. 

The  fact  that  0 < g^ ( X)  < * is  established  by  the 
induction  hypothesis,  (3.1.4),  and  the  fact  that  the  right 
hand  summation  of  (3.1.4)  is  non-empty  for  X ft  1. 

We  are  now  interested  in  higher  order  approxima- 
tions to  the  probability  that  the  system  works. 

Theorem  3.1  C.  Under  the  conditions  of  Theorem 
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3.1  A, 


iUk-lco®l-1(vlS-k|Cl®l«,®1 

k-0 


(3.1.6) 


where  9‘2)  (X)  can  be  determined  recursively  on  |C0(X)|  by 
the  equations 


n . 

- Z g (0.,1)  * and 

. . IT  1 ~ 

1*=1 


(X) 


ieC]^  (X) 
i=TT(OifX) 


(3.1.7) 


E....ui42,(li^)  - 


ieCQ(X) 


X ^ 1 . 


Proof: 


Equations 


(3.1.4)  and  (3.1.5)  can  be  combined  to 


give: 


i=ir (0ifX) 


r /I  U (l..X)l  " 

k Z p.[e  di»X)-k'  i'“ 

ieC0(X) 


k l 


X t 1 


ieC^ (X) 


(3.1.3) 
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We  prove  the  theorem  by  induction  on  Cq  (X)  . 

When  X ■ 1 use  the  fact  that  Ee_  (X)  ■ 1 and 
“ X * “ 

theorem  3.1  B to  establish  the  result. 

Now  assume  the  theorem  is  true  for  X such  that 
|Cq(X)|  - m.  We  wish  to  establish  that  the  theorem  is  true 
for  X such  that  |Cq(X)  | ■ m+1. 

Multiply  both  sides  of  (3.1.8)  by  and 

take  the  limit  as  k goes  to  zero.  Theorem  3.1  B and  the 
induction  hypothesis  then  give  the  desired  result  for  X such 
that  |Cq(X)  | - m+1.  End  of  proof  of  the  theorem. 

In  a similar  manner  one  can  prove  the  following 

theorem. 


Theorem  3.1  D.  Under  the  conditions  of  theorem 
3.1  A,  define  g^  (X)  ■ g^  (X) , then 


lim  k“lC°(£}  l"n‘(e1T(X)-klC0(-)  lg1j1)  (X) -. . . -k  I c°  (^J  *+m"1g1|m)  (x)  ] 
k+0 


TT 


g,Sm+1)  (x) 


(3.1.9) 


Where  g1|m+1)  (X)  can  be  determined  recursively  in 


terms  of  smaller  m and  | Cq  (X)  | as  follows 


(1) 


■ l g(m+l- | Cq (X) I ) 

X:0<  | Cq  (X)  | <m+l  71 


and 


l a(m+1)(X) 


iECi (X) 
i*TT(Oi,X) 


xtg‘ra) (Oi(x) 


^ uig<m+1)(i.,x)  - z u^^oo 

ieCQ(X)  1 1 “ ieC1 (X)  1 " “ 


X ¥ 1 


Proof:  The  proof  exactly  parallels  the  proof  of  theorem 

3.1  C and  is  omitted. 


3 . 2 Limiting  Optimal  Ergodic 
Unreliability  of  the  System 
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Theorem  3.2  A.  For  a given  X,  a candidate  policy 


n minimizes  g^X)  given  in  theorem  3.1  B over  possible 
candidate  policies  if  and  only  if  A^  A^^  V^Cg  (£)  for  all 
£ such  that  Cq  (j^)cCq  (X)  . 

Furthermore , 


( 


min  g^  (X)  = 
n 


ieC0(X) 


|c0  (X)  I , 

1 n XT 

permutations  j»l  g 

of  CQ(X) 


(3.2.1) 


where  A.*  = max 

J k e first  j 
elements  of 
the  permutation 


Proof ; Again  by  induction  on  | Cq  (X)  | . The  theorem  is  true 
vacuously  when  |Cg(X)|  = 0 or  |Cg(X)|  = 1.  Now  assume  the 
theorem  is  true  for  all  £ such  that  |Cq(£)|  * m. 

Let  X be  such  that  |Cg(X)  | = m+1.  Equation 
(3.1.4)  tells  us  that  g^X)  is  minimized  iff  A^  ^ > A^^ 
ieCg(X)  , and  g^d^X)  is  minimized  for  all  ieCQ  (X) . 

Therefore,  by  the  induction  hypothesis,  g^  (X)  is 
minimized  iff  A^  ( ) >A^VieCg (^)  for  all  £ s.t.  Cg  (£)  C 
C0(X>;  establishing  the  first  part  of  the  theorem. 

For  the  second  part,  note  that  (3.1.4)  tells  us 


iitfiaa 
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min  g^X) 

TT 


" ma7  -r;  . * /i  mtn  W*> 

jec0(x)  3 1ECo  — * 


Substituting  for  min  gn(l^#X)  by  the  induction  hypothesis 

TT 


gives 


T9*^  i |c0  (X)  1-1  x 

TT — = max-  'V.'  . rZ(yS  l . .n.  rr 

irr  m 1 -irr  m ^ i£Cn(X)  permutations  3*1  3 

ieCQ(X)  3eCQ(X)  0 Qf  c (X) -{i.} 


but  the  double  sum  simplifies  to  give: 


|c0 (X) |-1  , 

y n 

L . . 1 .* 
permutations  j=l  3 


of  CQ(X) 


and  we  also  obtain  by  definition 


max  A . 
jeCQ(X) 


|CQ(X)  | 


for  any  permutation,  and  thus  the  second  result  is  esta- 


blished. End  of  proof  of  theorem  3.2  A. 


Theorem  3.2  B.  Let  U{k)  be  the  optimal  ergodic 


unreliability  of  the  previously  described  system.  Let  m^ 
min  cardinality  of  the  cut  sets  * min  cardinality  of  the 


minimal  cut  sets.  Then 


lim  k U (k) 
k-*0 


where  is  defined  below. 


z n 

K:  K 

K is  a cut  set 
I K I ■ mi 


permutations  j»l 
of  K 


(3.2.5) 


where  ■ max 

ic  first  j elements  of  the  permutation 
Furthermore,  for  all  k < kg,  for  some  k^  > 0,  a 
policy  it  is  not  optimal  unless,  for  every  cut  set  K with 
| K | ■ and  every  £ such  that  CQ  (£)  c K,  Xfl  ^ j >_  XjVieCgfyJ 

Proof : Let  ir°  be  a candidate  policy  which  minimizes  all 

g^fx)  as  given  in  theorem  3.2  A.  Let  it*  be  a candidate 
policy  which  does  not  have  the  property  given  in  the  last 
paragraph  of  theorem  3.2  B.  Let  U°  (K)  be  the  ergodic 
unreliability  of  policy  tt°,  U' (K)  be  the  ergodic  unrelia- 
bility of  policy  tt'.  Then  by  theorem  3.2  A 


lim  k’miu°(k)  < lim  k"mlU' (k) 
k+0  k+0 


(3.2.6) 


and  thus  for  all  k < k(n’)  with  k (ir 1 ) > 0,  policy  tt • is  not 
optimal.  Since  the  number  of  candidate  policies  is  finite, 
the  last  paragraph  of  the  theorem  holds. 

Furthermore,  if  tt"  is  a candidate  policy  which 
minimizes  g^ (X)  for  all  Cq  (X)  which  are  cut  sets  of 
cardinality  m^,  then 


i |i|inii|iipw'Piui^w 


'lunftajwww^wwwiw1*  t *wijnfw».wywri 


AO 


lim  k"miU"(k)  = C,  (3.2.7) 

k-0  1 


where  is  given  by  (3.2.5). 

Thus  (3.2.4)  is  true  since  for  all  k < kg  the 
optimal  policy  must  be  of  the  form  given  for  tt",  of  which 
there  are  a finite  number.  End  of  proof  of  theorem  3.2  B. 


The  higher  order  approximations  of  theorem  3.1  D 
can  now  be  applied  to  further  narrow  the  possible  optimal 
policies  for  small  k.  Such  extensions  appear  quite  messy. 

It  appears  somewhat  easier  to  use  alternating 
renewal  theory  to  further  specify  the  optimal  policy  for 
small  k.  Such  an  approach  is  taken  in  section  3.3  which 
follows . 


tfaaiMiiinriYi 
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3.3  Limiting  Expected  Passage  Times 


Theorem  3.3  A.  For  the  previously  described 
system,  and  it  a candidate  policy,  let  T (k,X)  be  the  expec- 

7T  — 

ted  time  to  go  from  state  X under  policy  tt  to  the  1^  state. 
Then 


lim  T (k  ,X)  * T(0,X)  ■ Z t^ 
k-0  ~ ieCQ (X)  i 


< “> 


(3.3.1) 


Proof ; The  equations  for  the  expected  passage  times  yield 
solutions  continuous  in  k,  and  the  passage  time  in  the 
absence  of  failures  is  easy  to  compute  and  is  independent 
of  policy.  End  of  proof  of  Theorem  3.3  A. 


Theorem  3.3  B.  Let  Bn(k,X)  be  the  expected  time 
the  system  is  failed  during  passage  from  X to  1 under  policy 
tt.  Let  I (X)  be  the  minimum  number  of  component  failures  in 
X to  cause  system  failure.  Then 


lim  k1-I(^  B (k,X) 
k-0  * “ 


(3.3.2) 


Proof : The  proof  is  by  induction  on  I (X) . Theorem  3.3  A 

establishes  the  result  when  I (X)  a 0. 

Now  assume  that  the  theorem  is  true  for  all  X 
such  that  I (X)  * 0,1,..., m and  that 
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lim  k^”mB  (k , X)  = 0 

k-0 


(3.3.3) 


for  X such  that  I(X)  > m. 


The  following  identity  for  B^fkjX),  X 7*  1,  obtained 


by  conditioning  is  helpful. 


(X  , Y,  + k E U.)B(k,X) 
icci  (X)  1 ” " 


(3.3.4) 


60$(X)  + An  (X)  Btt  (k  ' 1tt  (X)  + kieC^(x)WiBTt(k'0i'- 

Here  6 is  the  Kronecker  delta,  6^  = 0,  i / j;  6^  = 1, 


i = 3 


(MX)  = 1. 


Now  let  X be  such  that  I (X)  > m+1,  which  implies 


We  must  show  that 


lim  k~mB  (k,X) 
k+0 


(3.3.5) 


and  our  inductive  proof  in  complete. 

Multiply  both  sides  of  (3.3.4)  by  k m,  and  let 
k-*0.  By  (3.3.3)  the  sum  on  the  left  hand  side  goes  to 
zero  as  k goes  to  zero.  We  also  have  that  * q ( X)  = T^e 
last  term  on  the  right  hand  side  has  zero  limit  since 
I(0i,X)  >_  I (X)  -1  = m,  and  (3.3.3).  We  will  thus  establish 
(3.3.5)  as  desired  if 


Um  k B„0c,1„(!1),  X)  = 0 


(3.3. 6) 


Note  that  Kljffx)'^)  1 KX)  * m+1  so  that  (3.3.6)  is  the 
same  type  of  statement  as  (3.3.5).  We  prove  both  of  these 
by  subinduction  on  |Cg(X)|. 

Bw(k,l)  = 0 so  that  (3.3.5)  is  true  when  Cq  (X)  * 

1.  If  (3.3.5)  is  true  for  |CQ(X)|  ■ j then  (3.3.6)  is  true 
for  |c0(ln ,X) | - j implying  (3.3.5)  is  true  for  |CQ(X)|  ■ 
j+1.  End  of  the  subinduction  and  end  of  inductive  proof  of 
Theorem  3.3  B. 


Theorem  3.3  C. 


lim  k"I(-)Bir(k,X) 
k-0 


dw<*> 


(3.3.7) 


where  0 < d^ (X)  < ®,  X / 1 . 


If  4>  (X)  = 1,  d^  (X)  are  determined  by 


X*(X)d7r(x)  “ 6I(X)  ,I(lir(x)  ,X)XTT(x)dir(1Tr(x) + 


(3.3.8) 


ieCi (X) 

1(0.  ,xf=I(X)-l 


^idir  (°i '— ) 


If  (j)  (X)  = 0,  d^  (X)  is  the  time  to  fix  the  system 


when  k = 0 . 


Proof ; The  theorem  is  proven  for  <J>  (X)  = 0 by  a continuity 
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argument  similar  to  that  used  in  proving  3.3  A. 

Now  assume  the  theorem  is  true  for  £ such  that 
I (yj  = l,...,m.  Let  I (X)  = m+1.  Multiply  both  sides  of 
(3.3.4)  by  )c“^rT1+1)  and  iet  k 0 . The  sum  on  the  left  hand 
side  goes  to  zero  by  Theorem  3.3  B.  The  first  term  on  the 
right  hand  side  is  zero,  and  the  third  term  on  the  right 
hand  side  has  a limit  equal  to  the  last  term  in  (3.3.8)  by 
the  induction  hypothesis,  and  Theorem  3.3  B.  If  I(l7t(X),X)> 
I (X)  the  second  term  on  the  right  hand  side  has  zero  limit. 

(3.3.8)  is  thus  established  for  a maximal  element 
of  { X | I ( X ) = m+l},  and  for  a maximal  element  of  the  set 
deleting  a maximal  element,  and  so  forth. 

Inductively,  d^fX)  < 00  and  does  not  equal  zero, 
since  the  last  term  on  the  right  hand  side  of  3.3.8  is  non- 
zero inductively. 
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3 . 4 Limiting  Optimal  State  Actions 
Under  First  Order  Passage  Times 

In  an  intuitive  sense,  we  wish  to  minimize  the 
expected  time  in  failed  states  during  passage  between  two 
states,  as  k goes  to  zero,  since  the  expected  passage  time 
becomes  independent  of  policy.  We  can  make  this  idea  more 
precise  using  alternating  renewal  theory. 

Theorem  3.4  A.  Suppose  that  for  two  candidate 
policies  tt°  and  tt1 

b (ergodic  probability  the  system  is  failed 
under  tt  ' ) 

(3.4.1) 

for  k < kp 
with  kg  > 0 , b > 0 

< c B ,(k,X)  for  k < kQ  , (3.4.2) 

with 

0 <_  c < 1,  k 0 > 0 

Then  as  k goes  to  zero,  it'  is  not  optimal. 

Proof : We  contradict  the  optimality  of  tt'  as  follows. 

Construct  a policy  tt"  which  uses  policy  tt®  between  X and  1^ 
and  uses  policy  tt'  between  1^  and  X. 


B^Ot/X) 
T7r0(k,X)  - 


and 

B^q (*,X) 
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Let  S(k,X)  be  the  expected  time  to  pass  from  to 

X under  policy  tt  ' , and  let  F(k,X)  be  the  expected  time  the 

system  is  failed  during  passage  from  1 to  X under  tt  * . By 

36  37 

alternating  renewal  theory,  ' the  ergodic  probability 
that  the  system  is  failed  under  tt  1 is 


P^, ( $ ( X ) — 0 ) 


, (k,X)+F(k,X) 
Tn,  (k,X)+S  (k,X) 


(3.4.3) 


and  the  ergodic  probability  that  the  system  is  failed 
under  tt  " is 


P7T„(0(X)=O) 


btt0  (k,X)+F(k,x) 
Tn0  (k  ,x)"+S(k  ,x) 


(3.4.4) 


Now  suppose  tt"  is  not  strictly  better  than  tt'  or  that 


P^ ,,  (<f>  (X)=0)  > P^,  (<MX)=0) 


(3.4.5) 


Substitutions  from  (3.4.3)  and  (3.4.4)  into  (3.4.5) 
and  suppression  of  the  arguments  gives 


T„0  iv  + 


<T,.+s>(B,o-V) 

(B„,+F) 


(3.4.6) 


using  (3.4.2)  in  (3.4.6)  gives 


•ttO  < v - <1  - 1)b„0-b“7 


(T^.+S) 
~+F~ 


(3.4.7) 


for  k < kg . 
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Using  (3.4.1)  and  (3.4.3)  in  (3.4.7)  we  obtain 

Tn0  i V - 

for  k < kQ , a contradiction  as  k -*•  0 since  lim  T q ■ 

k+0 

lim  T , , by  Theorem  3.3  A.  End  of  proof  of  Theorem  3.4  A. 

k-0 

Theorem  3.4  B.  A candidate  policy  n'  which  does 
not  minimize  the  d^lx)  of  Theorem  3.3  C is  not  optimal  for 
k < k'  for  some  k*  >0. 

Proof ; Suppose  d^tx)  < d^,  (X)  . Then  by  (3.3.7)  of  Theorem 
3.3  C 

Bu  n (k  t X)  < cB^  , (k  ,X)  for  c < 1, 

all  k < kg, 

satisfying  (3.4.2)  of  Theorem  3.4  A. 

By  Theorem  3.2  B,  if  n ' is  optimal  as  k goes  to 

zero  then 

lim  k“mi  (ergodic  probability  of  failure  under  tt  ' ) ■ c. 

k+0  1 

(3.4.9) 

where  m^  is  the  minimal  size  of  a cut. 

Theorem  3.3  C tells  us  that 

lim  k”1 ) B „ (k ,X)  - d „ (X)  . (3.4.10) 

k-*-0 


I 
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Since  I (X)  < ni^,  (3.4.9),  (3.4.10),  and  Theorem  3. 3 A imply 
condition  (3.4.1)  of  Theorem  3.4  A.  Theorem  3.4  A now  gives 
the  desired  result.  End  of  proof. 


We  now  use  Theorem  3.4  B and  equation  (3.3.8)  to 
find  many  of  the  optimal  actions  for  the  limiting  optimal 
policy  as  k goes  to  zero. 


When  <f>  (X)  = 0,  minimize  £ — r — over  sets  of  com- 

ieA  i 

ponents  A which,  when  fixed  in  state  X,  fix  the  system. 


One  of  these  components  must  be  under  repair  in  the  limiting 
optimal  policy.  Thus  rule  translates  to:  a failed  system 

should  be  repaired  in  such  a manner  that  the  time  to  repair 
in  the  absence  of  failures  is  minimized. 


Now  successively  minimize  d^^  in  the  following 


manner.  First  compute  d for  the  set  of  X with  I (X)  * 1. 


Choose  a maximal  element  in  this  set  and  minimize  according 
to  (3.3.8).  (The  optimal  action  is  to  repair  the  failed 
component  with  largest  in  this  case.)  Continue  to  choose 
maximal  elements  of  the  uncomputed  states  with  I(X)  = 1 


minimizing  according  to  (3.3.8)  until  the  minimal  d^'s  are 


computed  for  all  states  X,  with  I(X)  = 1.  Now  do  the  same 
procedure  with  the  set  of  states  with  I (X)  = 2,  then  with 
the  set  of  states  with  I (X)  = 3,  and  so  forth  until  d is 
minimized  for  all  states. 

One  easy  example  to  compute  is  the  parallel  system 
The  limiting  optimal  policy  for  small  k must  always  choose 
the  largest  repair  rate  failed  component  in  any  state. 


i.iniiii 
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CHAPTER  4 

Highly  Unreliable  Systems 

To  model  highly  unreliable  systems  we  may  take  the 
model  of  the  last  chapter  and  examine  the  behavior  as  k goes 
to  infinity. 

By  a time  rescaling  argument  such  a model  is  equi- 
valent to  a system  for  which  the  i—  component's  failure 

rate  is  u-  and  for  which  the  i—  component's  repair  rate  is 

\.  1 

i 

T * 

Thus , we  may  equivalently  consider  a model  for 

which  the  i—  component's  failure  rate  is  vu  , and  for  which 
f*  h 

the  i—  component's  repair  rate  is  kA^,  and  examine  the 
behavior  as  k goes  to  zero. 

Much  of  the  analysis  is  similar  to  that  of  the 
last  chapter,  and  we  shall  be  interested  in  the  approximate 
optimal  reliability  of  the  system  and  the  optimal  action  for 
small  k. 

Intuitively  the  results  for  highly  unreliable 
systems  are  of  less  usefulness  than  the  results  for  highly 
reliable  systems . 


4 . 1 Limiting  Ergodic  Probability  of 
States  for  Candidate  Policies 

For  a candidate  policy  n (see  section  1.4),  define 

L (X)  = minimum  possible  number  of  repairs  under 

tr  to  reach  X from  0 irrespective  of  (4.1.1) 

failures . 


Thus,  X = 0 is  the  only  state  X such  that  (X)  = 

0,  and  X = 1 , 0 is  the  only  state  X such  that  (X)  = 1. 

The  only  states  X with  L (X)  = 2 are  X = l_,n.,  1M 

tt  - - ttiu; 

0 and  X * 1 (1  o)'-* 

7TUtt(0) 

Theorem  4.1  A.  (X)  has  the  following  properties: 

(X)  _<  n . (4.1.2) 

(X)  > l Cx (X) | . (4.1.3) 

If  X > Y,  then  (X)  >_  (Y)  . (4.1.4) 

If  L (X)  = m > 0,  then  either  3 ieC^  (X)  s.t.  (0^  ,X)  =m-l, 

i=Tr  (0i,X) , 

or  3 ieCQ  (X)  s.t.  L^d^X)  = m . (4.1.5) 

If  L_  (X)  = m > 0,  then  3 ieC,  (X)  s.t.  L (0 • ,X)  < (4.1.6) 
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Proof : (4.1.2)  Any  state  may  be  reached  by  repairing  all 

components  and  then  allowing  components  in  Cq  (X)  to  fail. 

(4.1.3)  In  order  to  reach  X from  0,  every  compon- 
ent in  (X)  must  be  repaired  at  least  once. 

(4.1.4)  One  way  to  reach  Y _<  X is  to  reach  X 

first  and  then  allow  the  components  in  C^(X)  - (Y)  to  fail. 

(4.1.5)  Let  X be  such  that  L (X)  = m,  and  let 

7t  — 

9. 'X-i'^2  ' ’ ’ ’ be  the  successive  states  passed  through  in 

reaching  X from  0 using  m repairs.  Each  state  X^  is  obtained 
from  the  previous  state  by  one  repair  or  one  failure. 

At  least  one  such  path  is  possible  by  the  definition  of 


L (X) . 

IT  - 

We  have  two  possibilities.  Either  the  transition 
from  X^  to  X is  from  a repair,  or  the  transition  from  X^  to 
X is  from  a failure. 

In  the  first  case  (X^)  m-1,  since  we  have  a 

path  from  0 to  X^  with  m-1  repairs.  In  addition,  (X^)  >. 
m-1  since  the  existence  of  a path  to  X^  with  fewer  than  m-1 
repairs  implies  the  existence  of  a path  to  X with  fewer  than 
m repairs,  a contradiction.  Thus  L^tX^)  = m-1.  Choose 
i = C^X)  - C^(X^)  to  satisfy  the  first  condition  on  the 
right  hand  side  of  (4.1.5). 

If  the  transition  from  X^  to  X is  from  a failure, 
we  conclude  in  an  analogous  matter  than  L^X^)  = m.  Now 
choose  i = C^X^)  " C^X)  to  satisfy  the  second  condition  on 
the  right  hand  side  of  (4.1.5). 

(4.1.6)  Again  let  0 ,X^, . . . ,X  be  the  successive 
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states  passed  through  in  reaching  X from  0 using  m repairs. 
Let  the  last  repair  take  place  in  state  J^.  Then  by  the 
argument  used  in  the  proof  of  (4.1.5),  (X^)  = m-1. 

Let  i = tMx^),  then  ieC^(X),  since  if  not  by 
(4.1.4)  X J X^  =>  L^  (X)  _<m-l,  a contradiction. 

However  0^,X  _<  since  the  last  repair  (which  was 
component  i)  took  place  at  . Thus  (4.1.4)  tells  us 
( 0 i , X)  < m-1  and  (4.1.6)  is  proven. 

End  of  proof  of  Theorem  4.1  A. 


Theorem  4.1  B.  Let  the  failure  rate  of  the  i— 
component  be  y ^ , and  the  repair  rate  of  the  i—  component 
be  kA^  Let  it  be  a candidate  policy  and  en  (X)  be  the  ergo- 
dic  probability  of  state  X under  policy  it  (note  that  the 
dependence  on  k has  been  suppressed)  . Then 


lim  (k1"L7T(-))e  (X)  = 0 

k-*0  n 


(4.1.7) 


Proof : The  proof  is  by  induction  on  L^X).  Clearly  (4.1.7) 

holds  when  ( X)  = 0 since  e^ (X)  is  bounded.  Now  assume 

that  (4.1.7)  is  true  'for  X such  that  L (X)  = m < n. 

— TT  — 

Equating  the  ergodic  rate  of  entering  and  leaving 
the  group  of  states  Gm  = {X  | L^  (X)  _<  m}  yields  the  following 
equation. 


X:Ln(X)>m 


eir<*> 


ieC^X) 

Ln  x # 2L^  — m 


(4.1.8) 


X:L  (X)=m 

L (1  " ,X) >m 

IT  IT  (X)  - 


kXTT(X)e7T  (-) 


We  note  that  the  restrictions  on  the  summations 
in  (4.1.8)  are  correct  as  follows.  (4.1.4)  tells  us  that 
Gm  can  be  entered  only  through  failures,  and  left  only 
through  repairs.  Also  (X)  m,  (1^  ^ ,X)  > m (X)  = 

m. 

Now  multiply  both  sides  of  (4.1.8)  by  k m.  By  the 
induction  hypothesis  the  limit  of  the  right  hand  side  as  k 
goes  to  zero  is  zero.  Since  all  terms  on  the  left  hand  side 
are  positive,  they  must  all  have  zero  limit.  (4.1.6)  tells 
us  that  every  X such  that  (X)  = m+1  has  a term  on  the  left 
hand  side  which  is  a non-zero  constant  coefficient  multi- 
plied by  k“me  (X) . We  have  thus  proven  (4.1.7)  by  induction. 
End  of  proof  of  Theorem  4.1  B. 


Theorem  4.1  C.  Under  the  conditions  of  Theorem 


4.1  B 


lim  k“LTT(^)e_(X) 
k+0 


fTr<*> 


(4.1.9) 


where  0 < f (X)  < °°.  Furthermore,  f ^ (0)  = 1 and  for  X 1 0 


--  «■ ■— 


( £ p . ) f_  (x) 

i'C^X) 


ieCi  (X) 
i=ir(Oi,X) 

L (0.#X)-L  (X)-l 


^ i f tt  < 0 i ^ 


(4.1.9) 


ieC0 (X) 


Mif 7T  di#-)  * 


Equation  (4.1.9)  allows  computation  of  f (X)  recur- 
sively starting  with  the  lowest  values  of  (X)  and  maximal 


elements  of  sets  of  constant  L (X)  . 

71  — 


Proof : The  proof  again  is  by  induction  on  . (4.1.7)  tells 

us  that  lim  e (X)  =0,  X ^ 0,  thus  lim  e (0)  = 1,  establish- 
k->0  k-0 

ing  the  theorem  when  = 0. 

Now  assume  the  theorem  is  true  for  all  X s.t. 

L (X)  < m < n.  Write  the  balance  equation  for  ew  (X)  as 

71  — — Tl  — 

follows : 


(i^(xyi  + kWV*> 


(4.1.10) 


l kA.e  (0.,X) 
ieC^  (X) 
i=TT(0i,X) 


£ Mieff(li»X) 

iec:0(x) 


Assume  (X)  = m+1 . Multiply  both  sides  of 

.1.10)  by  k”  . By  Theorem  4.1  B,  the  second  term  on 


the  left  hand  side  has  zero  limit  as  k goes  to  zero.  Now 
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consider  the  first  sum  on  the  right  hand  side,  i = tt(o^,X) 
=>LTT(Oi,X)  > m,  so  that  by  Theorem  4.1  B a term  in  this  sum 
will  have  xero  limit  unless  L^tO^X)  = m.  In  this  case  we 
know  the  limit  by  the  induction  hypothesis.  Now  consider 
the  second  sum  on  the  right  hand  side.  Since  L^d^X)  > 
Ln(X)  by  (4.1.4),  Ln(l.,X)  > m+1.  Unless  L^d^X)  » m+1 
a term  in  this  sum  will  have  zero  limit  by  Theorem  4.1  B. 

Now  let  Xq  be  a maximal  element  of  (xlL^fX)  = m+1). 
A maximal  element  of  sets  of  binary  n vectors  always  exists. 
By  the  reasoning  of  the  previous  paragraph  the  theorem  is 
true  for  X^.  Non-zero  of  f^X^)  follows  from  (4.1.5). 

Let  X,  be  a maximal  element  of  {x|L  (X)  = m+1}  - 
{Xg}.  By  the  same  reasoning  the  theorem  is  true  for  X ^ , 
and  eventually  for  every  element  of  (xlL^X)  = m+l>.  End 
of  induction,  and  end  of  proof  of  Theorem  4.1  C. 

Note  that  higher  order  approximations  similar  to 
those  of  Theorem  3.1  D may  be  obtained  in  a similar  manner 
if  desired.  The  statement  and  proof  are  omitted. 


I]1""»l'^^;'  L ijij,),,  ,,,w.  , ,lllly.;TO;,W 
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4 . 2 Limiting  Optimal  Ergodic 
Reliability  of  the  System 

Theorem  4.2  A.  Let  R(k)  be  the  optimal  reliability 
of  the  previously  described  system.  Let  m^  = min  cardinality 
of  the  path  sets  = min  cardinality  of  the  min  path  sets. 

Then 


lim  k"m°R(k)  = C , 


(4.2.1) 


k-0 


whe  re 


C0  * 


m. 


max  1 (p  . ) (p  , +p.  T ~ 

(ll'***'1mn)  xl  xl  x2 


is  a path  set 


(u . + . . . +p . y 

Xl  xmQ 


) 


(4.2.2) 


without  loss  of  generality  the  elements  i^,^,.  • * 'imQ  may 

be  arranged  so  that  y.  < y.  <...  < y . 

il  - i2  - - imo 

Furthermore,  for  all  k > kQ  for  some  kg  > 0 any 

candidate  policy  tt  ' which  does  not  repair  ifi+1  in  the  state 

1.  .1.  ,...,1.  ,0Vn  <mn-l  for  some  set  (i i ) which 
xl  x2  1n  “ ” 0 1 ro0 

achieves  the  maximum  in  (4.2.2)  is  not  optimal. 

Proof : Let  it0  be  a policy  which  repairs  i in  the  state 

1.  ,1.  , . . . ,1.  , 0 V n _<  mQ-l,  for  a set  {in , . . . ,im  } which 


11  x2 


V - 


maximizes  (4.2.2). 


Then  it  has  no  working  states  X with  L^o  (X)  < mg 

by  (4.1.3),  and  exactly  one  working  state  X with  L^qW  * n^( 


Ifcr  n . 
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the  state  1.  ,...,1.  ,0  , since  any  other  states  X with 

xi  % - 

L 0<X)  = iHq  must  have  (X)  < m^. 

Let  R®(k)  be  the  ergodic  reliability  of  the  system 
under  policy  it®.  By  Theorem  4.1  C and  the  last  paragraph 
together  with  the  fact  that  the  number  of  states  is  finite, 
we  obtain 

limk"m°R°(k)  = f_0(lH  »0)  . (4.2.3) 

k-*-0  1 1m0 

But  (4.1.9)  yields 

f jj.()  (1^#»»»»1^  ,0)  a Cq  , (4.2.4) 

1 mQ 

where  CQ  is  given  by  (4.2.2). 

The  policy  it'  can  have  no  working  states  X with 

L^  | (X)  <mQ,  and  at  most  one  working  state  X with  L^,  (X)  = mQ. 

This  will  be  the  case  if  it  mimics  policy  it®,  but  for  a path 

set  (i. , . . . ,i  } which  does  not  maximize  the  expression  in 
1 mQ 

(4.2.2).  Regardless,  we  conclude  from  Theorem  4.1  C that  if 
R'  (k)  is  the  ergodic  reliability  of  policy  tt', 

lim  k"m0Rt  oo  <r  . (4.2.5) 

k-0  u 

Comparison  of  (4.2.5)  and  (4.2.3)  shows  that  ir ' 
is  not  optimal  for  k <kg. 

The  theorem  follows  from  the  fact  that  the  number 
of  candidate  policies  is  finite.  End  of  proof  of  Theorem  4.2  A. 
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4 . 3 Limiting  Expected  Passage  Times 

Some  of  the  terminology  in  section  4.3  will  be 
identical  with  terminology  in  section  3.3  although  having  a 
different  meaning.  Chapters  3 and  4 are  independent  and 
no  confusion  should  arise  for  the  reader. 

Theorem  4.3  A.  For  the  system  described  in 
Theorem  4.1  B,  let  T^fk/X)  be  the  expected  time  to  go  from 
state  X to  state  0 under  candidate  policy  tt.  Then 

lim  T (k,X)  = T ( 0 , X ) < » . (4.3.1) 

k-0 

Proof : The  equations  for  the  expected  passage  times  yield 

solutions  continuous  in  k.  However,  the  passage  time  to  0 
in  the  absence  of  repairs  is  finite  and  independent  of  policy. 
End  of  proof  of  Theorem  4.3  A. 

Theorem  4.3  B.  Let  G^kjX)  be  the  expected  time 
the  system  is  working  during  passage  from  X to  0 under  candi- 
date policy  tt.  G (k,0)  = 0.  Let  K (X)  be  the  minimum  i.  amber 

IT  — TT  — 

of  repairs  under  policy  tt  (allowing  any  number  of  failures) 
to  reach  a working  state  from  X without  passing  through  0_. 

We  make  the  convention  that  <f>(X)  = 1 — > K (X)  = 0.  Then 

— TT  — 

lim  k1_KTT  (^g_  (k,X)  = 0 . (4.3.2) 

k+0 


|| ....... 


■JkL*y» 
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Proof ; The  proof  is  by  induction  on  (X) . Theorem  4.3  A 


establishes  (4.3.2)  when  K (X)  ■ 0. 

IT  — 

Now  assume  (4.3.2)  is  true  for  X such  that  (X)  _< 
m and  assume  that  if  K (X)  > m,  then 

7T  — — 


lim  k1_mG  (k,X)  = 0 

k-*-0 


(4.3.3) 


The  following  identity  is  easily  obtained  in  the 


standard  manner.  If  X ^ 0, 


< = **i  + kWVk'» 


ieC^X) 


(4.3.4) 


6i<fr(X)  + ieCJ(X),1iGlj(k#0i'-)  + kXTT(X)GTI(k,1TT(X) 


Now  suppose  (X)  ^ m+1  which  implies  <MX)  **  0. 
Multiply  both  sides  of  (4.3.4)  by  k’m.  The  induction  hypo- 
thesis (4.3.3)  tells  us  the  second  term  on  the  left  hand 
side  has  limit  zero  as  k goes  to  zero.  We  note  in  general 
Ktt  ^it  (x)  - K.W-1,  since  one  possible  way  to  reach  a 

given  working  state  from  X is  to  go  through  dfl(x)'— 
Specifically  Kn  (1^  ^ ,X)  ^ mf  and  the  induction  hypothesis 
(4.3.3)  tells  us  the  last  term  on  the  right  hand  side  has 
limit  zero. 

We  note  that  K^tO^X)  > Kf  (X)  , so  that  (4.3.4)  and 
the  arguments  of  the  previous  chapter  show  that  if  Xg  is  a 


T 
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minimal  element  of  (xjK^X)  ^ m+1)  - {0}  then 

lim  k_mG_(k#X«)  = 0. 
k-0 

Similarly  we  establish  the  same  result  for  a minimal  element 
of  (x|K  (X)  > m+1}  - {0}  - {X-}  and  so  forth  for  all  X with 
(x)  >_  m+1.  End  of  the  induction  proof  and  end  of  the 
proof  of  Theorem  4.3  B. 

Theorem  4.3  C. 

lim  k“K"  (-)GTr(k/X)  = C^X)  (4.3.5) 

where  0 <C7f(X)  < °°,  X ? 0. 

For  X such  that  4>  (X)  = 0, 

+ 

(4.3.6) 


Proof ; The  theorem  is  proven  for  (X)  = 0 by  a continuity 
argument  similar  to  that  used  in  Theorem  4.3  A.  The  rest 
of  the  proof  is  based  on  (4.3.4)  and  is  similar  to  previous 
proofs  of  this  type  given  and  is  omitted.  End  of  proof  of 
Theorem  4.3  C. 


( E P^C^X) 


ieCi  (X) 


ieCi (X) 

Ktt  (0i /X)  =Kir  (X) 


SK w (X)  .Kw(l,(x)  -X)+l  X,r<5)C,'(1,,(X) 


. ..-.-.  im... 


m 


4 . 4 Limiting  Optimal  State  Actions 
Under  First  Order  Passage  Times 
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Theorem  4.4  A.  Suppose  that  for  two  candidate 
policies  it®  and  tt  ' 

G 0(k,X) 

b (ergodic  probability  the  (4.4.1) 

system  works  under  tt') 

for  k < kQ  with  kQ  > 0,  b > 0,  and 

GirO(k'*>  >yG7r,(k,X)  (4.4.2) 

for  k < kQ  and  y > 1. 

Then  as  k goes  to  zero,  tt  • is  not  optimal. 

Proof ; We  contradict  the  optimality  of  tt’  as  follows. 

Construct  a policy  tt"  which  uses  policy  tt®  between  X and  1^ 

and  uses  tt  1 between  1 and  X. 

Let  S(k,X)  be  the  expected  time  to  pass  from  1 to 

X under  policy  n',  and  let  F(k,X)  be  the  expected  time  the 

system  works  during  passage  from  1 to  X under  policy  tt'.  By 

36  37 

alternating  renewal  theory,  ' the  ergodic  probability  that 
the  system  is  working  under  w*  is 

G , (k ,X)  + F (k,X) 

P . (<MX)=1)  - — (4.4.3) 

* T , (k,X)  + S(k,X) 
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and  the  ergodic  probability  that  the  system  works  under 
policy  it"  is 


p „(<MX)  = 1) 

TT  — 


(4.4.4) 


G^otkrX)  + F(k.X) 
\0(k,X)  + S (k,  X) 


Now  suppose  that  it"  is  not  strictly  better  than 
it ' , or  that 


= 1)  < 


P*.  (♦(*)  = 1) 


(4.4.5) 


Substitution  from  (4.4.3)  and  (4.4.4)  into  (4.4.5)  and 
suppression  of  the  arguments  gives 


63 


(T-.+S) 

(G.O  - Gw.) 


T’°  1 T"  + -* 


(4.4.6) 


Using  (4.4.2)  gives 


<V+S) 


r,o  1 V + <1'y)G.o  — 

' IT  * 


(4.4.7) 


+F) 


for  k < kn . 


Now 


, using  (4.4.1)  and  (4.4.3) » we  obtain 


T^o  l V + b(1’7)T^ 


(4.4.8) 


for  k < kg , a contradiction  as  k + 0 since 


lim  ttto  " }iTH 
k+0  k+O 

by  Theorem  4.3  A.  End  of  proof  of  Theorem  4.4  A. 

Theorem  4.4  B.  A candidate  policy  »’  such  that 
K^.tt)  1 min  K^X)  is  not  optimal  for  k < k'  for  some  k'  > 0 
Furthermore , a candidate  policy  n ' such  that 


C¥, (x)  + 

is  not  optimal  for 


max  [C_0  M 1 

ti°:K  o (X)  »minK  (X) 
n " IT 


k < k'  for  some  k'  > 0. 
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Proof : Compare  tt ' with  it®  where  tt®  has  both  of  the  above 

properties.  By  (4.3.5)  we  have  that 

lim  k‘Kir0  (-G7t0(X)  = C^qCX)  (4.4.9) 

But  K^q  _<  iHq  = min  cardinality  of  path  sets. 
Therefore,  (4.4.9),  (4.2.1),  and  (4.3.1)  imply  condition 
(4.4.1)  of  Theorem  4.4  A. 

Also  Theorem  4.3  C implies  condition  (4.4.2)  of 
Theorem  4.4  A,  and  the  remainder  of  the  theorem  follows  from 
4.4  A.  End  of  proof  of  Theorem  4.4  B. 

Theorem  4.4  B and  (4.3.6)  give  us  a valuable 
technique  to  solve  for  the  limiting  optimal  policy. 
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DIVISION  III 
Specific  Cases 

In  the  next  two  chapters  we  treat  various  examples 
of  specific  structure  functions  and  or  specific  repair  or 
failure  rates. 

The  two  component  parallel  system  shows  that,  in 
general,  such  specific  cases  are  extremely  complicated. 

This  system  has  a symmetric  structure  function  of  only  two 
components,  and  there  are  only  two  candidate  policies  to 
consider.  Nevertheless,  the  region  of  optimality  for  each 
candidate  policy  is  somewhat  complicated. 

Let  be  the  candidate  policy  which  repairs  com- 
ponent 1 when  both  components  are  failed,  and  let  t*ie 

candidate  policy  which  repairs  component  2 when  both  compon- 
ents are  failed. 

If  is  the  ergodic  probability  of  (i,j)  under 
policy  1,  then  by  standard  methods  P^j  can  be  computed  from 
the  following  equations: 


(lil+M2)Pu 

” X2P10+X1P01 

<WP10 

= li2p11+X1p00 

(Xl+y2)P01 

■ >11PU 

X1P00 

■ U1P10+U2P01 

1 1 J ' IflHWS 
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P 4.  p + p + p 

00  01  *01  *11 


The  solution  to  the  previous  equations  is: 


00 


01 


10 


11 


d"lwlM2  (yl+M2+Al+A2) 


d‘\AlX2 


d_1M2Xi  ^i‘f’^2+Xl) 


d‘1A1A2 (y2+Ax) 


(5.0.2) 


where 

2 

d=  (^1+A1) (u2A1+u2X2+u1M2+y2  +AlA2) 

The  ergodic  probabilities  under  policy  tt2  are  ob- 
tained by  symmetry  by  exchange  of  the  subscripts  in  the  last 
equations,  rr^  is  easily  seen  to  be  optimal  if 


yly2Al+M2Al  +y2  Al+Al  A2 


Mly2A2+MlA22+yl2A2+A22xi 


(5.0.3) 


a criterion  not  expressible  in  the  form:  f(y^,A^)  g(y2#X2) 

The  results  from  Chapters  3 and  4 tell  us  that  for 
highly  reliable  systems  we  should  repair  the  component  with 
largest  first,  for  highly  unreliable  systems  we  should 
repair  the  component  with 


- < • •••••• i 
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largest  ratio  — . 

wi 

One  result  we  can  easily  prove  is  the  following: 

A2 

if  a)  X.  > A-  and  b)  rr-  > rr-  * then  for  the  two  component 

X — z V»  2^  M 2 

parallel  system  it  is  optimal  to  repair  the  first  component 
first. 

Proof : 


by  inequality  b;  and  is  >_ 

U1M2|W1(SJ)+X2  (nj)+X2+X2(C“>  (5j)' 

by  inequality  a, 


and  thus  is  optimal  by  (5.0.3). 

Whether  or  not  the  fact  that  the  same  action  is 
optimal  for  both  highly  reliable  and  highly  unreliable  systems 
implies  it  is  also  optimal  for  all  values  of  the  parameter 
k given  in  chapters  3 and  4 is  an  open  question. 
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CHAPTER  5 
The  Series  System 

The  structure  function  for  the  series  system  is 
4>  (x)  = nxi.  We  note  that  there  is  only  one  state  vector  X 
for  which  4»  (X)  = 1,  this  is  X = 1.  Since  the  dwell  time  in 
the  working  state  and  transitions  to  other  states  are  inde- 
pendent of  policy,  alternating  renewal  theory  tells  us  that 
maximizing  the  ergodic  probability  of  the  working  state  is 
equivalent  to  minimizing  the  expected  time  until  the  working 


state  is  reached. 


T 
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5 . 1 Series  System  with  = y 

We  first  consider  the  case  where  the  failure  rates 
are  identical,  \i ^ - y,  for  every  i.  We  arrive  at  the  sur- 
prising conclusion  that  the  policy  doesn't  matter  1 The 
heuristic  reasoning  that  all  components  must  be  repaired 
anyway,  so  that  the  policy  obviously  doesn't  matter;  is 
spurious  because  components  are  subjected  to  failure  during 
the  course  of  repair  of  other  components,  and  will  be  contra 
dieted  when  the  failure  rates  are  not  identical. 


Theorem  5.1  A.  Let  v be  a candidate  policy  (see 
section  1.4),  and  let  T^ (X)  be  the  random  variable  which  is 
the  rime  to  reach  state  1 from  state  X under  policy  tt.  Then 


E(e-8T"<*>] 


L (X) 

rro 


(5.1.1) 


where 


L(X) 


1 + S l 

iec^X) 


X7  + s(s+M)  } T7X7 

i i<j  i j 

if jeC1 (X) 


+ ...  + 
(5.1.2) 


s (s+y) 


(s+  ( | c.  (x)  1 — i) m ) n 

A ieC1(X) 


1 

r: 

i 


Proof:  Let  L^X)  = Ele“sT"*-  ]. 

We  obtain  the  following  set  of  equations  for  X / 1 
by  standard  techniques. ^ 


(X7T(X)  + lCl(^  lM  + S)LTT(^ 


(5.1.3) 


A iv\ ^i»  tv\  ^ M ^ Lf(0.,X)  . 

1T(X)  IT  Tt(X)  - jeCl(x)  11  3 " 


These  equations,  with  the  boundary  condition 


(1)  = 1 uniquely  determine  (X)  . We  show  that  L(X)  given 
by  (5.1.2)  satisfies  (5.1.3),  thus  establishing  (5.1.1). 

It  is  readily  seen  from  (5.1.2)  that  for  X 1, 


\(x)iL<Mx)'?y  - L<*>i 


S + S (s  + u)  £ t — + . . . + 

• /.*\  A • 


(5.1.4) 


ieC^X)  i 


s(s+y) . . . (s+|c, (x) lu)  n ~ 

ieC^X)  i 


It  also  follows  from  (5.1.2)  that 


l L ( 0 • , X ) = |C,(X)|  + 

jeC^X)  3 


( | C.  (X)  I -1) s Z yi  + (|c.  (X)  |-2)S(S+W)  l T-T-  + 

1 " iec.(x)  Ai  1 " i< j Vj 

1 i,  jeCL  (X) 


...  + s (s+Vi)  . . . (s+(|c.  (X)  ( — 2 ) w ) I A. 

* • -/i  /i/\  A 


ieC1 (X) 


n a . 

jeC1(X)  3 


(5.1.5) 


■ | 


and  (5.1.2),  (5.1.5),  and  (5.1.4)  check  when  substituted  in 


1 


are 


(5.1.3)  since  the  factors  which  multiply 
found  to  be  equal  by  the  following  readily  ~ verified 

identity . 

( |C. (X) | u+s) (s (s+m) . . . (s+ (m-1) u) ] = 

1 (5.1.6) 

s (s+u) . . . (s+mp)  + M 1 1 C1 (X)  | -m] s (s+u) . . . (s+ (m-1) y) 

End  of  proof  of  Theorem  5.1  A. 


We  note  that  the  previous  theorem  is  valid  also 
for  simultaneous  effort  or  mixed  policies,  provided  full  use 
is  made  of  the  repairman.  The  proof  is  similar. 

Note  that  Theorem  5 . 1 A says  that  the  time  to  re- 
pair the  system  is  stochastically  independent  of  the  candi- 
date policy,  and  thus  the  expected  time  to  repair  the  system 
is  independent  of  candidate  policy.  We  conclude,  by  alter- 
nating renewal  theory  that  the  ergodic  probability  that  the 
system  works  is  independent  of  candidate  policy. 

We  also  easily  obtain  by  standard  methods  the  ex- 
pected time  to  fix  the  system: 


. -L*(X)L(1)+L(X)L'  (1) 

EtT^(X>l  = gl,  (X)  | 

s=o  (L(i)r 


(5.1.7) 


or  that 
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E [T^  (X)  ] 


z rr  + 
(iWcjx)  i 


L 

i<  j 
{i, ji^c1(x) 


V 

rrr 

i D 


(5.1.8) 


rrrx 


2!  Z 

i<j<k  "i"j"k 
(i  / j ,k)pc1  (X) 


(n-1)  iy 

FT 


n-1 


The  ergodic  probability  that  the  system  works 
under  any  candidate  policy  is  readily  available  through  al- 
ternating renewal  theory,  or  from  the  ergodic  probability 
of  the  time  reversible  policy  whose  potentials  are  given  in 
(2.2.2).  Let  e(lj  be  the  ergodic  probability  of  1 under  any 
candidate  policy.  Then 


e (1)  = 


1+Zt^  +2!  I 


rrr 

i i 


i<  j 


X7T7 

i 1 


n 

+ n!  n (jil) 


(5.1.9) 


i=l 


We  can  also  easily  compute 


F(tt,s,1) 


V 


e~SS(X(t))dt|x(0)  = l)dt 

J0 


9 


the  expected  integral  of  the  discounted  time  that  the  system 
works,  by  an  alternating  renewal  theory  approach.  The  ex- 
pected value  of  this  integral  for  the  first  working  period 
is 


°°  -nyt.l,.  -st..,. 
nye  [-(1-e  )]dt 

0 S 


1 


ny+s 


The  expected  discounted  time  until  the  system  begins  its 
second  working  period  is  E(e-sTiL)  ■ 


! n UO.,1) 
n r 1 v 1 1 

ny+s^n.^  Ml)  J 


(n-2) s (s+y) l y y ■ + 

i < j i j 


nu+s)L(l)  t"+ (n-1) s I j-  * 

i ^ 

. + s (s+y) . . . (s+  (n-2)  y)  jpjj—  1 

j j 


We  thus  obtain  by  alternating  renewal  theory 


F(tt,s,1 ) = 


+ E(e'sTii)F(w,s,l) 


We  conclude  that: 


F(ir,s,l) 


Ml) 


s+s  (s+y)  l r^-  + s (s+y)  (s+2y)  1 t— r—  +. . .+s  (s+y)  . . . (s+ny)  jjy- 
i i<j  i j i i 


and  that 


F(tt,s,X) 


MX)  ' 
L ( 1) 


F(7r,S,l) 


(5.1.12) 
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5 . 2 The  Two  Component  Series  System 


We  now  relax  the  assumption  that  the  failure  rates 
of  all  components  are  identical.  We  start  with  a two  com- 
ponent series  system. 

Again,  there  are  only  two  candidate  policies  to 
consider.  Let  be  the  candidate  policy  which  repairs  com- 
ponent 1 when  both  components  are  failed,  and  define  ir2 
similarly.  If  t^(X)  denotes  t.he  expected  time  to  go  from  X 
to  1^  under  policy  tt^,  then  we  obtain  easily: 


tx ( (0, 0)  ) 


— + tl((l,0)) 


^((1,0)) 


1 

A2+lJl 


+ 


\2*Vl 


t1((0,0)) 


(5.2.1) 


which  has  the  solution 


^((0,0)) 


t2( (0,0) ) 


Wui 

Xl»2 


Xl+A2+M2 

*1*2 


(5.2.2) 

9 


where  t2  is  obtained  by  interchanging  the  subscripts  in  the 
expression  for  t^  by  symmetry.  Thus  is  superior  to  n2 
if  y^  < y2'  a relatively  simple  criterion.  It  is  somewhat 
surprising  that  the  repair  rates  are  irrelevant  for  deter- 
mination of  the  optimal  policy. 


'WVMY.'jJKVT'anT- ;’7^r 
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We  are  now  interested  in  the  distribution  of  time 
to  reach  (1,1)  from  (0,0)  under  policy  »r  We  proceed  in 
the  following  standard  manner.  Make  the  state  (1,1)  an 
absorbing  state.  The  system  evolution  is  then  governed  by 
a continuous  time  Markov  chain  with  the  following  transition 
24,37 

diagram. 


of  the  eigenvalues24  are  then  r ^ and 


The  negatives 


r2  where 


rx  = 1/2  [(x^x^)  + /(Xj+Xj+u^f  -4x1x2i 


(5.2.3) 


r = 1/2  [ (X1+X2+M1)  “ /(Xj^+Xj+MjT  "4X1X21 


The  system  evolution  conditioned  on  X(0)  = 0 is 


► 

P{X(t)  = (0,0)} 

1 

X,  -r-  _r  t ri"Xl  -r7t 

1 2 e rl  + zr~~T  e 2 

rl"r2  'l  2 

P{X(t)  = (1,0)} 

-X1  .-nt  * X1  e-^1 

ri-r2  rrr2 

* 

- 

Uiyd 
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Thus  if  Xn 


is  the  random  variable  denoting  the 


passage 


time 


from  (0,0)  to  (1,1)  under  policy  then 


-r 


Fn(t) 


rrr2 


1-  e'1^  + 


i-  e'1^ 


rl-r2 


(5.2.5) 


where  F12(t)  " p^xll 

Note  that  F21(t)  is  also  determined  by  (5.2.5) 


where  r,  and  r,  are  determined  from  (5.2.3)  by  replacing  H 
with  Thus,  regarding  Fu<t)  and  rx  and  r2  as  functions 


of 


F21(t)  = Fu(t)  + 


3F11(t) 


dy-. 


(5.2.6) 


Theorem 


< X,,  , or 


5.2  A.  If  yx  £ w2  then  XH  s-t  21 ' 


equivalently  Fn<t)  « T2l(t>  V t > 0.  Furthermore,  if 
Ul  < u2,  then  Fu(t)  < F21(t)  V t > 0.  « •>!  - h2  . 


X11  St  x2l* 


3F 


^ jtj.  > 0 for  t > 0. 


Proof;  We  use  (5.2.6)  and  show  that  ^ 

We  first  note  that  from  (5.2.3)  r^  = X1A2"  so 

that 


3r, 

4 

3y- 


r,  By, 


(5.2.7) 


Taking  the 


partial  derivative  of  Fu<t),  we  obtain  by 


uww* 


1 1"  i,  a iuMun*  m 


.......  . Ml  ,i.|  T- 
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the  chain  rule: 


3F..(t>  3P,,(t)  3rL  3Fn(t)  3tj 

l1  — « * + “ dpi 


T7 


3y. 


3r. 


(5.2.8) 


substituting  from  (5.2.5)  and  (5.2.7)  yields: 


3y  i 


rrlV 

L _ 

3lll  (r^rj) 


Fll(t)  . !fl  L— 1.  i2-2e<rrr2lt+(r1-r2)t(l+e,rl'  i’  )) 

2 


tr1_r2>t: 

(5.2.9) 


3r. 


r o 71  that  — — — >0.  we  conclude 

We  note  from  (5.2.3)  tnat 


that  all  the  terms  outside  the  square  brackets  are  strictly 


.ositive.  we  note  that  the  term  in  the  square  brackets  can 


be  written  as  (2-2eX  + X(l+eX)l  where  X - (rrr2)t  > 0 if 

v V ...  e / v\  * ( 


t > 0.  Let  f (X)  - 2-2eX  * X(l*eX)  . We  will  show  f(X)  > 0, 


I > o,  and  thus  the  theorem  will  be  proven  as  desired.  Note 

;hat  f (0)  - 0,  f (0)  - 0,  r<X>  > 0,  x > 0,  and  the  desired 

42  j 


result  follows  from  Taylor's  theorem  with  remainder, 
of  proof  of  Theorem  5.2  A. 


End 
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5.3  The  "Lazy"  Repairman  Two 


Component  Series  System 


In  this  section  we  improve  the  results  of  the  last 
section.  Specifically,  we  assume  that  the  repairman  is 
interrupted  periodically  and  does  no  work  on  the  system. 
These  interruptions  occur  independently  of  the  state  of  the 
system.  Of  course,  failures  in  working  components  can  occur 
during  the  interruptions.  We  show  that,  in  this  more  general 
setting,  if  < y 2 the  system  is  repaired  stochastically 
more  quickly  by  using  policy  ir^  while  the  repairman  works 
than  by  using  policy  tt  2 while  the  repairman  works.  If  y^  = 
y2  the  time  under  both  policies  is  stochastically  equal. 

We  then  have  the  following  model . The  repairman 
works  for  the  first  time  period  of  length  t^,  is  idle  for 
the  second  time  period  of  length  t2,  works  for  the  third  time 
period  of  length  t3,  is  idle  for  the  fourth  time  period  of 
length  t^,  and  etc. 

The  evolution  of  the  system  state  during  odd 

periods  and  under  policy  ir  ^ is  governed  by  a continuous  time 

24  37 

Markov  chain  with  the  following  transition  diagram,  ' 


(0,0) 


(1,0) 


-Jy 


(1,1) 


and  the  evolution  of  the  system  state  during  even  time 
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periods  is  governed  by  a continuous  time  Markov  chain  with 
the  following  transition  diagram: 


(0,0) 


(1,0) 


(1,1) 


Theorem  5.3  A.  Under  the  system  evolution  pre- 
viously described,  in  which  the  repairman  alternately  works 
and  is  idle,  the  time  to  reach  (1,1)  from  (0,0)  under  policy 
of  section  5.2  is  strictly  stochastically  less  than  the 
time  to  reach  (1,1)  from  (0,0)  under  policy  tt2  if  < U2- 
If  m ! = p2,  the  times  are  stochastically  identical. 

Proof:  We  show  that  the  probability  that  the  system  has 

been  repaired  at  any  time  using  policy  is  greater  than 
the  probability  the  system  has  been  repaired  at  the  same 
time  using  policy  tt2.  It  is  easily  seen  that  we  need  only 
consider  odd  numbered  time  intervals,  since  repairs  cannot 
occur  during  the  even  numbered  intervals.  We  show  that  this 
is  true  for  period  2n-l,  regardless  of  length,  by  induction 
on  n.  Theorem  5.2  A shows  that  the  claim  is  true  for  n = 1 . 

We  now  assume  that  the  theorem  is  true  for  period 
2n-l,  and  wish  to  show  its  truth  for  period  2n+l.  At  the 
end  of  period  2n-l,  the  probability  vector  for  the  system 
state  under  is 


(5.3. 


_n-l 


P (X(  £ t.  ) = (0,0)  ) 

*1  " i=l 


2n-l 


p (X(  £ t.)  = (lfO) ) 

*1  " i=l 


II 

pu 

P12 

. 

for  some  P-^  and  p12- 

Similarly  under  tt2,  the  probability  vector 
for  the  system  state  is 


* 

f 

2n-l 

P (X ( £ t.)  = (0,0)) 

*2  " i-1 

= 

P21 

2n-l 

p (X(  £ t.  ) = (0 ,1) ) 

V2  ' i-1  1 

P22 

for  some  P2^  and  p22* 

At  the  end  of  period  2n,  the  appropriate  state 
vectors  are 


• 

2n 

P +P,«(l-e  ^lt2n) 

p (X(  I t . ) = (0  » 0 ) ) 
*1  “ i=l  1 

ur  12 

2n 

p e-Ult2n 

p (X ( £ t.)-(lfO)) 

7T  i _ -i 

P12e 
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2n 

P (X  ( E t . ) = (0 ,0)  ) 
2 i=l 

er 

P +P  (l-e”^2^2n\ 
*21**22  U e ’ 

2n 

P_  (X(  £ t . )*  (0 ,1) ) 
2 i*l 

p e”^2^2n 
22® 

• 4 

Now  introduce  the  following  notation. 


Fn(t)  = P (repair  time  is  greater  than  t from  (0,0) 
under  with  repairman  always  working) 

F12  (t)  = P (repair  time  is  greater  than  t from  (1,0) 
under  ir^  with  repairman  always  working) 

f2i  (t)  = P (repair  time  is  greater  than  t from  (0,0) 
under  ir2  with  repairman  always  working) 

F22(t)  = P (repair  time  is  greater  than  t from  (0,1) 
under  *2  with  repairman  always  working) 


(5.3.5) 


Let  t be  the  elapsed  time  in  period  2n+l.  Letting 
F^tt)  be  the  probability  the  system  is  not  yet  repaired  at 
this  time  under  policy  tt^,  and  F2  (t)  be  the  probability  the 
system  is  not  yet  repaired  at  this  time  under  policy  n2, 
and  using  (5.3.3),  (5.3.4),  and  (5.3.5)  we  obtain 


fi(t)  = lPn+p12(1'e"Wlt2n)  ]Fn(t)  + pi2e"Mlt2nFi2(t) 

(5.3.6) 
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and 


F2(t)  = tp2l+P22  (l'e"M2t2n)]F21(t)  * P22e~M2t2nF22(t)  * 

(5.3.7) 

Note  that  by  induction  hypothesis  if  t2n  = 0,  then 
F^t)  <_  F2  (t)  since  period  2n+l  is  then  just  a continuation 
of  period  2n-l.  Again  equality  occurs  iff  u1  = u2* 

Now  let  t~  •+  ® in  (5.3.6)  and  (5.3.7).  We  note 
2n 

by  the  induction  hypothesis  that  P11  + P12  - P21  * P22 
since  these  represent  the  respective  probabilities  that  re- 
pair occurs  after  the  end  of  period  2n-l  under  the  two 
policies.  Again  equality  occurs  iff  = W2.  The  fact  that 
F11  ( t)  iF21^  was  shown  in  s®ction  5.2,  30  that  as  t2n 
00 , Fx(t)  < F2(t)  . 

Thus,  for  t2n  * 0 or  t2n  -*•  00 , F^  (t)  F2  (t)  , with 
equality  iff  y^  = u2-  We  show  that  this  implies  there  is  no 
value  of  t2n  such  that  F^(t)  > F2  (t) . 

Taking  derivatives  of  (5.3.6)  and  (5.3.7)  yields: 


3Fx(t) 


Tt 


2n 


= lJ1p12lFil(t)"F12(t)]e"yit2n 


and 


8F2(t)  - - -uot 

= M2P22 [F21(t)"F22 (t) le 


Tt 


2n 


(5.3.7) 


(5.3.8) 


We  see  that  by  (5.3.5)  F^(t)  > F^2  (t)  and 


t 

{ 


J I ' 1 ■ ' ..  ~ 1 


F2i  (t)  > f22  (t)  so  that 
*F, (t) 

A V A 


3F2  (t) 


Now  assume  there  is  a value  of  t2n*  such  that  when 
t',  (t ) - F2(t)  >0.  We  show  this  leads  to  a contra- 


diction. 


Consider  the  above  expression  as  a function  of 


fc2n* 


G(t2n)  “ Fl(t>  “ F2(t)  (5.3.10) 

Taking  the  derivative  of  (5.3.10)  with  respect  to 
t2n  and  using  (5.3.8)  and  (5.3.9)  we  obtain 

G' (t2n)  " kie’yit2n  ~ k2e’M2t2n  (5.3.11) 

where 

kl  " wlP12[Fll(t>  '*  Fi2(t)]  > ; 

k2  “ W2P22^F21^  ” F22^fc^  > 0 

Since  G(0)  _<  0 and  G(t')  > 0 we  require  for  some 
tQ  <t',  G'  (tQ)  > 0.  But  (5.3.11)  requires  that  G'  (t)  > 0, 

t > t'  since  y^  _<  We  conclude  that  G(t)  > 0,  t > t', 

a contradiction  as  t + ». 

If  y1  < y2»  by  the  same  reasoning  we  cannot  have 
G(t')  >0  since  G(0)  <0. 


Stochastic  equality  of  the  repair  times  follows 
when  |i j.  - U2  since  F^t)  _<  Fj(t)  and  Fj(t)  < ^(t).  End  of 
inductive  proof,  and  end  of  the  proof  of  Theorem  5.3  A. 

Corollary!  When  the  periods  t.  are  random  variables  inde- 
pendent of  the  state  of  the  system,  Theorem  5.3  A still 

holds . 


■■  - wfazimn..  &■' 


5.4 


The  n Component  Series  System 
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The  results  of  the  last  section  intuitively  tell 
us  that  the  optimal  policy  for  an  n component  series  system 
is  to  repair  the  components  in  order  of  increasing  Un- 

fortunately, this  general  result  seems  difficult  to  prove. 

We  can  prove  the  following  result. 

Theorem  5.4  A.  Suppose  n is  a candidate  policy 
in  the  form  of  a list;  i.e.,  suppose  policy  tr  always  repairs 
the  failed  component  closest  to  the  top  of  some  fixed  list 
of  the  components.  If  the  components  on  the  list  are  not 
arranged  in  order  of  increasing  failure  rates,  then  tt  is  not 
optimal. 

Proof ; Find  two  components  on  the  list  for  tt  in  which  the 
failure  rates  are  inverted.  Call  these  components  i and  j, 

Mi  > Mj. 

Repair  of  the  system  from  0 under  n will  take  the 
following  form.  First,  in  phase  1,  repair  all  the  components 
on  the  list  above  i and  j.  Then,  in  phase  2,  repair  compon- 
ents i and  j , interrupting  the  repair  of  i and  j to  repair 
the  set  of  components  above  i and  j on  the  list  if  any  should 
fail.  Lastly,  in  phase  3,  repair  the  reot  of  the  components, 
interrupting  if  necessary. 

However,  by  Theorem  5.3  A we  know  that  the  second 
phase  (repairing  i and  j with  interruptions)  can  be  done 
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strictly  faster  if  the  order  of  repair  of  these  two  compon- 
ents is  switched. 

Thus  a policy  it  ' which  uses  tt  during  phases  1 and 
3 and  switches  the  order  of  repair  of  components  i and  j 
during  phase  2 will  have  an  expected  system  repair  time 
strictly  less  than  the  expected  system  repair  time  under 
policy  tt  , contradicting  the  optimality  of  tt.  End  of  proof 
of  Theorem  5.4  A. 

Theorem  5.4  B.  If  an  n component  series  system 
whose  repairman  is  subject  to  the  kind  of  interruptions 
described  in  section  5.3  always  has  a component,  i,  which 
may  be  optimally  repaired  whenever  failed,  then  the  optimal 
series  policy  is  in  the  form  of  a list. 

Proof : The  proof  is  by  induction  on  n.  The  theorem  is 

clearly  true  for  n = 2.  For  an  n+1  component  system,  note 
that  under  a policy  with  component  i always  repaired,  if 
possible,  the  repairman's  availability  to  the  n other  compon- 
ents follows  the  interrupted  repairman  model  of  the  last 
section.  Thus,  if  the  theorem  is  true  for  n components,  it 
is  also  true  for  n-rl  components . End  of  proof  of  Theorem 
5.4  B. 


Intuitively,  the  complete  symmetry  of  the  series 
system  suggests  that  a list  policy  is  optimal.  However,  we 
will  see  in  section  6.3  that  optimal  policies  for  general 


CHAPTER  6 


Stochastically  Identical  Components 

When  all  components  are  stochastically  identical, 
i.e.,  A . = A,  = u;  certain  symmetries  in  the  structure 
function  can  often  be  exploited  to  show  that  certain  actions 
are  not  optimal.  Thus,  by  contradiction,  the  optimal  action 
in  a given  state  can  often  be  obtained. 
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Permutation  Operators 


When  all  components  are  stochastically  identical# 
the  proper  system  evolution  is  maintained  whenever  working 
components  are  permuted#  providing  that  the  permutations 
depend  only  on  the  past  history  of  the  system.  Similarly, 
the  proper  system  evolution  under  a different  policy  is  ob- 
tained if  non-working  components  are  permuted,  provided  that 
permutations  depend  only  on  the  past  history  of  the  system. 

We  are  particularly  interested  in  permutations 
which  take  place  at  system  state  transitions. 


Let  the  system  evolution  under  a candidate  policy 
tt  be  on  a probability  space  (ft,?,P)  . Thus,  given  weft  we 
know  X(t,iD)  for  all  t >_  0 . Without  loss  of  generality,  let 
the  n—  transition  for  X(t,w)  occur  at  tn  (u>)  with  0 
tQ  ( oj  ) < t ^ (tu ) < t2  (w)  < t^  (<d)  < ...  . This  is  possible 

since  the  measure  of  the  set  of  points  u>  without  this 
property  is  zero.  Let  the  total  number  of  transitions  up  to 
and  including  time  t be  N(t,w). 

A permutation  operator,  p,  will  permute  the  com- 
ponents of  an  n vector.  Thus  P can  be  specified  by  a per- 
mutation of  the  first  n integers,  p = (n^,...nn)  , n^  ^ n^  , 
i ft  j ; and  n^e(l,...,n)  with  the  understanding  that  (pX)^  = 
Xn . . Thus , if  p = (2,1,3)  , P (X^ , X2 # X^ ) = (X2  # # X^) • 

The  identity  permutation  operator,  I,  will  be 
defined  to  be  I = (l,2,...,n). 

For  a permutation  operator  p,  and  an  integer  i, 
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1 <_  i <_  n define  p ( i ) = (p  (1^,0) ) . 

We  are  particularly  interested  in  a new  system 
evolution  X* , whose  transitions  occur  at  the  same  times  as 
the  times  of  transition  for  X,  and  whose  state  at  a time  of 
transition,  X' (u>»  t^  (u>) ) , is  obtained  by  permutation  of 
X (uj , t^  (w)  ) , the  original  state  vector  at  the  same  time. 

Given  (w) , i = 0 , 1 , . . . , permutation  operators, 

define  the  permutation  operator  pn (o)  as  follows: 

pnM  = pn  pn_i  (w)  • * * po  (“)  * (6.1.1) 

By  convention  p — 1 ( oj ) = I.  Now  define  the  permuted  process 
X'  (w  , t)  as  follows  : 

X'  (w,t)  = pn  (w)  X(uj,tn  (uj)  ) , tn(w)  <_  t < tn+1(t°)  • (6.1.2) 

Theorem  6.1  A . Let  the  system  evolution  under 
candidate  policy  i be  on  a space  (n,?,P)  with  the  properties 
previously  described  in  this  section.  Let  p^(o>)  depend  only 
on  X(u>,t)  for  t < t^(w)  , with  the  property  that 
p0  (w)  X (u)  ,0)  = X (b> , 0 ) , and  for  i = 0,...  , 

Pi  + 1 (w)  p^  (ui)  X (w  ,ti  + i (oj)  ) = p1  (w)  X (w,t^  + 1 (co)  ) . (6.1.3) 

Then  X' (ui,t)  given  by  (6.1.2)  gives  the  proper  system  evolu- 
tion on  (ft,?,P)  for  some  non-stationary  policy  which  depends 
on  the  past  history  of  the  system. 


I 


Proof:  Let  the  component  under  repair  under  n ' at  (w,t) 


when  t^  (w)  < t < t^  ( | (oi)  be  constant  and  be  tlio  component 
which  is  constantly  under  repair  under  it  in  the  same  inter- 
val, permuted  by  p1  (w)  . This  is  just  p1  (u>)  [it  ( X (w  , t^  (w)  ) ) ) . 

The  proper  future  system  evolution  under  v ' , given 
the  past  history  of  the  system,  follows  immediately  from 
(6.1.2),  and  from  the  proper  future  system  evolution  of  X 
given  the  past  history  of  X guaranteed  by  the  probability 
space  (fl,?,P)  . 

More  explicitly,  a repair  in  the  original  process 
is  accompanied  by  a repair  in  the  primed  process,  and  the 
failure  of  a particular  component  j in  the  original  process 
is  accompanied  by  the  failure  of  the  working  component  in 
the  primed  process  associated  with  component  j through 
permutation.  The  stochastic  identity  of  the  components 
insures  that  the  proper  law  is  followed  for  the  primed 
process.  End  of  proof  of  Theorem  6.1  A. 

As  an  example  of  the  use  of  Theorem  6.1  A,  we  show 
how  the  system  evolution  under  all  candidate  policies  can 
be  mapped  into  the  same  probability  space. 

Let  the  system  evolution  under  a candidate  policy 
ir  be  on  the  space  (fi,?,P)  previously  described.  We  show  how 
to  obtain  p^ (w)  so  that  the  system  evolution  under  another 
candidate  policy  tt ' is  given  by  (6.1.2). 

For  X(w,t^  (o>) ) = 1,  Pi(w)  = I.  For  X(w,t^(w))  f 1, 
and  i = 0,1,...  , let  p^(w)  be  the  permutation  operator  which 
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exchanyes  the  coordinates  p 1 ^ (to)  [n  (X(u>#t^  (to)  ) ) ] and 
7i 1 (p1”^  (uj)  X (a,  t^  ( to ) ) ] . By  construction,  p ^ (to ) exchanyes 
coordinates  which  are  zero  in  p1  (w,t^  (ou ) ) , and  thus 

(6.1.3)  is  satisfied. 

By  the  argument  given  in  the  proof  of  Theorem  6.1  A, 
the  component  under  repair  under  tt  ' for  (w, t)  , t ^ (w)  <_  t < 
ti+i  (a>)  , is  ( uj)  ( uj ) (ti  (x  (to , t) ) ) . By  the  construction  of 
p^(ui),  this  is  just  tt  ' [ p 1 ^ (ui)  X (w , t^  (w) ) ] . However,  by 
(6.1.2)  and  (6.1.3),  this  is  just  tt  ' (X’  (w , t) ) . 

Thus,  by  Theorem  6.1  A,  X'(u>,t)  given  by  (6.1.2) 
is  the  proper  system  evolution  under  candidate  policy  tt  ' . 

Notice  that  the  above  argument  is  correct  even  if 
P^(oj)  permutes  coordinates  which  are  equal  to  1 in 
pi_1  (m)  X (uj,  ti  (u) ) . 
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6.2  Contradiction  of  Optimal.  Policies 

The  primary  use  of  Theorem  6.1  A is  to  contradict 
the  optimality  of  candidate  policies. 

Assume  candidate  policy  Tt  is  optimal.  We  attempt 
to  choose  the  p^  (to)  in  such  a manner  that  the  evolution 
under  (6.1.2)  of  X'  strictly  dominates  the  evolution  under 
the  original  policy  of  X.  That  is,  we  choose  p^(uj)  in 
Theorem  6.1  A so  that  if  4>(X((o,t))  = 1,  then  <p  (X 1 (u> , t) ) = 1 
and  that  on  a set  of  non-zero  measure  in  w and  t, 

4> (x1  (u>,  t) ) = 1 and  <$>(X(io,t))  = 0.  This,  of  course,  implies 
the  existence  of  a policy  tt  ' which  is  strictly  better  than 
tt,  and  thus  contradicts  the  optimality  of  tt. 


6.3 


An  Example 


In  the  following  example  we  show  the  use  of  the 
previously  described  method  for  determination  of  the  optimal 

pol icy . 

Consider  a four  component  system  whose  structure 
function  is  given  by: 


<M(x1,x2,x3,x4))  = l-U-x^)  d-x3x4) 

Such  a structure  function  may  be  represented  pictonally  as 
a network  as  follows: 

1 2 
r 0- 0 , 


l o 0 1 

3 4 

with  the  interpretation  that  the  system  works  iff  there  is 
a path  from  A to  B through  working  components. 

Intuitively,  due  to  the  symmetry  present,  the 
optimal  policy  is  of  the  following  form:  repair  any  failed 

component  when  none  or  two  of  the  components  work.  When  one 
component  works , repair  the  component  whose  repair  causes 
the  system  to  work.  For  other  states  there  is  no  choice 

allowed  in  a candidate  policy. 

To  prove  that  such  a policy  is  optimal,  we  contra- 

diet  the  optimality  of  any  candidate  policy  not  of  the  given 
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form.  Ail  candidate  policies  of  the  given  form  are  then 
easily  seen  to  have  stochastically  equivalent  values  of 

We  now  show  that  the  optimal  policy  cannot  repair 
component  3 in  the  state  (1,0, 0,0). 

Let  it  be  a candidate  policy  which  repairs  compon- 
ent 3 in  the  state  (1,0, 0,0).  Let  the  evolution  under  it  be 
described  on  a probability  space  (0,?,P).  Let  n'  be  a 
candidate  policy  of  the  type  which  was  described  to  be 
intuitively  optimal. 

Use  the  method  described  at  the  end  of  section  6.1 
to  map  the  system  evolution  under  policy  tt * into  (ft,?,P)  . 
Remember  that  p i (u»)  may  arbitrarily  permute  working  coordin- 
ates of  p1”1  (id)  X(w, (w) ) . We  now  give  a rule  for  permuting 
such  coordinates. 

If  X1(w,ti(w))  = X2  (u> , (u>) ) = 1,  choose  p^w) 

such  that  either 

p.(u))p1_1(u>)  (1,1, 0,0)  « (1,1, 0,0) 
or 

Pi(w)p1“1(w)  (1,1,0,0)  - (0,0, 1,1) 

If  x3  (tu , t i (u>) ) = X4(w,ti(u)))  = 1,  choose  p ^ (u>) 
such  that  either 

Pi(uj)p1"1(w)  (0,0, 1,1)  = (1,1, 0,0) 
or 

pi(a))pi“1(w)  (0,0, 1,1)  = (0,0, 1,1) 
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We  have  yet  to  show  that  it  is  always  possible  to 


so  choose  (w)  . 


Theorem  6.3  A . It  is  always  possible  to  choose 
p.  (u)  as  described  immediately  previously,  and  <t>  (X'  (to,  (u>)  ) 

> $(X  (o),  t . (u)  ) for  every  i. 

Proof : The  proof  is  by  induction  on  i.  For  i = 0,  X' (u,0)  = 

X(w,0)  and  pg (w)  need  not  permute  working  coordinates  of 
X(to,0)  so  that  the  theorem  holds  in  this  case. 

Now  assume  the  theorem  holds  for  i.  We  show  that 
the  theorem  is  true  for  i+1. 

The  transition  at  t^  + 1(to)  can  either  be  a repair 
or  a failure.  If  the  transition  at  t^+1(w)  is  a repair  and 
4>(X  (ui,  ti  + 1 (w)  ) = 1,  then  | C 1 (X  (w,  t^  + 1 (uo)  ) ) | >_  2 and  thus 
| C-^  (X ' (td,  t^+^  (uj)  ) ) | > 2 and  by  the  nature  of  n ' , 

$ (X ' (w , t^  + 1 (to)  ))  = 1.  If  the  transition  at  is  a 

failure  and  4>  (X  (u>,t^  + ^ (to) ) ) = 1 then  by  construction  of  p^, 
either  X|  (w  , ti  + 1 (w)  ) = X^  (to , ti+1  (to)  ) = 1,  or  Xj  (to , ti+1  (o>)  ) = 
X^  (to,  ti+i  (w)  ) = 1;  so  that  <J>  (X  ' (to  , t^  + 1 (to)  ) = 1. 

s t 

Thus , regardless  of  the  type  of  the  i+1 — transi- 

t 

tion,  <f>  (X*  (to,  ti+1  (u>) ) = 1 if  0 (x (w  , ti+1  (to) ) = 1.  This 
condition  assures  domination  of  the  primed  process  over  the 
unprimed  process  at  (to , t^+^  (to)  ) and  also  ensures  that 
p i_^_  1 ( to ) can  be  chosen  as  desired.  End  of  induction. 

Strict  domination  follows  when  repairs  occur  for 
X = (1,0, 0,0).  End  of  the  proof  of  theorem  6.3  A. 


w****"”*"'”* 


T*-'  »f*f!  WhI Wrt 
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Corollary ; A candidate  policy  which  does  not  repair  compon- 
ent 2 in  state  (1,0,0, 0)  is  not  optimal. 

This  follows  from  the  last  theorem  and  the 
symmetry  of  components  3 and  4. 

Corollary : The  intuitive  optimal  policy  described  earlier 

is  optimal. 

This  follows  from  the  last  corollary,  the  symmetry 
of  components,  and  the  easily  proven  fact  that  all  candidate 
policies  of  the  intuitively  optimal  form  can  be  mapped  into 
the  same  probability  space  with  pointwise  equivalent  values 
of  $.  Such  a proof  is  similar  to  the  proof  of  Theorem  6.3  A, 
and  is  omitted. 

We  note  that  the  optimal  policy  for  the  system 
described  in  this  section  contradicts  the  hypothesis  that 
all  optimal  policies  can  be  expressed  in  the  form  of  a list 
(see  section  5.4). 

When  components  2 and  4 are  failed  in  the  state 
(1,0, 0,0)  it  is  strictly  optimal  to  repair  component  2. 

When  components  2 and  4 are  failed  in  the  state  (0,0, 1,0) 
it  is  strictly  optimal  to  repair  component  4. 
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6 . 4 A Second  Example 


The  previous  kind  of  argument  does  not  always 
find  the  optimal  actions  for  all  states  of  an  arbitrary 
structure  function.  Often,  a substantial  number  of  candi- 
date policies  can  be  eliminated  and  the  solution  for  systems 
can  be  considerably  simplified. 

For  example,  consider  the  following  5 component 

system 


with  the  following  structure  function 


<J>  ( (X1,X2,X3,X4,X5)  ) = 1-(1-X1X2)  (1-X3X4X5)  . (6.4.1) 

By  arguments  based  on  the  symmetry  similar  to 
those  given  in  the  last  section,  the  state  of  the  system  can 
be  summarized  by  the  following  two  coordinates, 

(Xi+X2,  X^+X4+X^) . In  the  same  manner,  there  are  only  two 
possible  actions  to  consider  in  any  state  which  are  1)  repair 
one  of  the  first  two  components,  and  2)  repair  one  of  the 
last  three  components.  We  can  also  prove  by  arguments 
similar  to  those  of  the  last  section  that  the  following 
actions  are  optimal. 
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State 


Action 


(0,0) 

(1,0) 

(2,0) 

(0,1) 

(1,1) 

(2,1) 

(0,2) 

(1,2) 

(2,2) 

(0,3) 

(1.3) 

(2.3) 


N.A. 


Thus,  the  only  state  in  which  the  optimal  action 
is  not  readily  obtainable  is  (0,2).  This  is  because  action 
2 dominates  action  1 for  small  numbers  of  transitions  where- 
as action  1 dominates  action  2 for  some  larger  numbers  of 
transitions. 

Nevertheless,  we  need  only  compare  2 candidate 
policies.  After  much  algebra,  we  find  that  action  2 is 
optimal  in  state  (0,2)  regardless  of  X and  p.  (This  is  the 
highly  reliable  and  highly  unreliable  optimal  action.) 

There  are  H j ] candidate  policies  to  consider 
j = 2 

for  an  n component  system.  When  n = 5 this  number  is 
5.45.3IO.25  a an(j  thus  we  have  provided  considerable 

simplification . 
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